0

I was wondering if there is really a need to use for example threading.Lock when working with Pythons sockets where each thread only sends and receives from that socket. Opening and closing is always handled by the parent.

When searching for an answer most people simply say that sockets are not thread safe and one needs to use something to serialize them. But nobody really explains as to why this is required.

Others say that send and recv are thread safe on a operating system level and thus may be used parallel without serialization (here). I don't know if I am correct here but I thought that Python uses the C implementation of POSIX sockets, correct? (What about the Windows implementation of sockets?)

If it not correct that send and recv are called without locks then why is that exactly? May it be possible that on thread receives the data that was requested by another?

ap0
  • 1,083
  • 1
  • 12
  • 37

1 Answers1

2

There's no reason why you must synchronize calls to send or recv as long as there is only a single sender or receiver. Consider if you had two threads attempting to receive a message. Let's say, for the sake of discussion, that a client=>server message is 8 bytes long with the first 4 bytes being a command and the second 4 bytes being an object identifier of some sort:

Thread A: sock.recv(8) obtains first 4 bytes
Thread B: sock.recv(8) obtains second 4 bytes

Here, neither thread ends up with a complete message. Now, many times that won't happen and you'd get the entire 8 byte message as a unit. But that is not guaranteed, and when the server / network get busy and OS network buffers fill up and you have multiple threads vying for CPU, it is more likely for this to happen. (And it's not just python BTW; the same would be true of a C program.)

So, yes, sockets are "thread-safe" in that exactly one thread's recv will obtain a given byte of data and there's nothing to prevent both threads from calling recv at the same time. The OS won't get confused by this. But there's nothing in a TCP (SOCK_STREAM) socket that will ensure that an entire "message" (of any length greater than a single byte) is received by one thread.

If you have two threads, with one only receiving and one only sending, then there is no competition and no locking would be required to maintain a coherent data stream.

UDP sockets OTOH are "message-oriented" so there isn't the same problem for them. Each send or recv delivers an indivisible datagram. A receiving thread won't get part of a datagram. It gets the whole datagram or nothing (though the datagram can be truncated due to insufficient buffer space; but in that case the remainder is simply dropped not delivered to the next receiving thread).

Gil Hamilton
  • 11,973
  • 28
  • 51
  • Thank you for the great explanation. Now I get the problem. And since all my threads are only reading from a socket I will have to synchronize them. – ap0 Dec 11 '18 at 12:34