When you make a connection to another machine over the network, you have a source port and a destination port.
The source port can be any port, meaning it doesn't have to match the destination port.
So while I can listen on port 80, I can connect to another system on port 80 using a local port of, say, 45000. In fact, most of the time the system will allocate a source port (also called a bind port) for you automatically when calling connect().
Needless to say, you don't have to listen on a port in order to connect to another system on that port, hence why TeamViewer - or whatever other application - can listen to port 80 on your machine without hindering your browser from making requests to other systems that are also listening to port 80.