Stop using TeamViewer. If you can’t setup your own secure self hosted remote desktop, then at least use AnyDesk.
I’m not claiming they’re perfect, or that any SaaS RD provider is good, but TeamViewer is right there with LogMeIn as the worst of a bad bunch.
Others may have better, or fancier solutions, but I’m a fan VPN -> Home Network -> VNC over SSH/TLS for Linux boxes, and RDP for Windows.
Again, none of VNC or RDP ports or services are ever exposed externally, and even on the LAN, they require authentication and use secure tunnels.
Full disclosure, I haven’t used RDP in a while and I don’t know what version of SSL/TLS it comes with anymore.
I know their are self-hosted AnyDesk style options and maybe they’re better than my approach, but I’ve never used them so I can’t really speak on that.