2

Suppose:

class A(object):
  def __init__(self):
    self.cnt = 0
  def __enter__(self):
    self.cnt += 1
  def __exit__(self, exc_type, exc_value, traceback)
    self.cnt -= 1
  1. Is it possible that self.cnt += 1 might be executed twice when multi-threading?
  2. Is it possible that for the same context manager instance, in multithreading, somehow __enter__ be called twice and __exit__ be called only once, so the self.cnt final result is 1?
martineau
  • 119,623
  • 25
  • 170
  • 301
est
  • 11,429
  • 14
  • 70
  • 118
  • What do you mean by same instance in multi-threading? Regardless if you give each thread its own instance of whatever contextmanager you might be using this wouldn't be relevant. If the data accessed is being shared between threads, access to them should be managed using mutexes/locks. – metatoaster Jun 29 '16 at 01:57
  • @metatoaster I agree, each thread has its own contextmanager, but my colleague says `__enter__` is just an instance method anyway, it might be called twice, so it's not thread safe, so the same contextmanager instance's `__enter__` is not thread safe. I feel lost – est Jun 29 '16 at 02:01
  • 1
    If each thread has its own instance, then there is no shared data. – martineau Jun 29 '16 at 02:12
  • What do you mean by being called twice? Do you mean by two different threads calling once each thus it has been called twice? – metatoaster Jun 29 '16 at 02:28

1 Answers1

3

No, thread safety can only be guaranteed through locks.

Is it possible that self.cnt += 1 might be executed twice when multi-threading?

If you have two threads running that, it will be executed twice. Three threads, thrice, etc. I am not sure what you really mean by this, perhaps show us how you are building/executing these threads with relation to your context manager.

Is it possible that for the same context manager instance, in multithreading, somehow __enter__ be called twice and __exit__ be called only once, so the self.cnt final result is 1?

Yes, final result can be non-zero, but not through the mechanism that you are assuming with asymmetric calling of enter and exit. If you use the same context manager instance across multiple threads, you can construct a simple example that can reproduce errors like so:

from threading import Thread

class Context(object):
    def __init__(self):
        self.cnt = 0
    def __enter__(self):
        self.cnt += 1
    def __exit__(self, exc_type, exc_value, traceback):
        self.cnt -= 1

shared_context = Context()

def run(thread_id):
    with shared_context:
        print('enter: shared_context.cnt = %d, thread_id = %d' % (
            shared_context.cnt, thread_id))
        print('exit: shared_context.cnt = %d, thread_id = %d' % (
            shared_context.cnt, thread_id))

threads = [Thread(target=run, args=(i,)) for i in range(1000)]

# Start all threads
for t in threads:
    t.start()

# Wait for all threads to finish before printing the final cnt
for t in threads:
    t.join()

print(shared_context.cnt)

You will inevitably find that the final shared_context.cnt often do not end up back at 0, even though when all the threads have started and finished with the exact same code, even though enter and exit have all been called more or less in pairs:

enter: shared_context.cnt = 3, thread_id = 998
exit: shared_context.cnt = 3, thread_id = 998
enter: shared_context.cnt = 3, thread_id = 999
exit: shared_context.cnt = 3, thread_id = 999
2
...
enter: shared_context.cnt = 0, thread_id = 998
exit: shared_context.cnt = 0, thread_id = 998
 enter: shared_context.cnt = 1, thread_id = 999
exit: shared_context.cnt = 0, thread_id = 999
-1

This is mostly caused by the += operator being resolved to four opcodes and only individual opcodes are guaranteed to be safe if only by the GIL. More details can be found at this question: Is the += operator thread-safe in Python?

Community
  • 1
  • 1
metatoaster
  • 17,419
  • 5
  • 55
  • 66
  • If we do not use shared context, change `with shared_context:` to `with Context()`, could cnt still be larger than `1` ? – est Jun 29 '16 at 03:25
  • Another question is with your example code, will the final result (I mean `joinall()`) of `shared_context.cnt` still guaranteed to be zero? – est Jun 29 '16 at 03:27
  • @est: you can just change it and see for yourself. If you used `with Context()` then within the code at hand there will be no shared data between threads. There is no `joinall()` call (unless you mean the for all threads, join them), but the results against the `shared_context.cnt` I have included show that it's not guaranteed to be zero - it could be _anything_. – metatoaster Jun 29 '16 at 03:32
  • I dont think `.cnt` could be anything, because `__enter__` and `__exit__` *are* symmetric, in the end after all threads finished, `.cnt.`would equals to exactly zero, right? – est Jun 29 '16 at 09:57
  • Well, run my code (multiple times) and find out if you don't believe my example outputs. As there are no locks guarding the operation that modified `.cnt` in such a non-thread-safe way there is always a chance that it will not reflect reality, and one of the outcome is that it is not zero at the correct time - where one thread assigned its new value to `.cnt` after a different thread had read but not assigned its value. – metatoaster Jun 29 '16 at 09:59
  • Thanks, I have ran it few times and still everytime the final result is zero. – est Jun 29 '16 at 14:17
  • Ah, you may have been using Python 3 which may have made changes to how threading may work. Still there is no guarantee that the intended correct result will be reproduced exactly the same across all Python implementations and their versions when proper thread safety is disregarded. – metatoaster Jun 29 '16 at 14:53