How would you make this Python Dictionary thread-safe?

Question

I have a Web Server running in Python. The server is private, so i only expect around 20 users to connect to it. The server is multi-threaded (8 cores at the moment, so 8 threads I guessed).

When requests come in, I am able to identify the users. On some queries, I need to update a simple dictionary of the form username -> Boolean. How could I make this one thread safe ?

Are you implementing the server yourself or are you using something like django, pyramid or twisted? — Dirk, Dec 13 '11 at 11:00

score 15 · Answer 1 · answered Dec 13 '11 at 10:46

15

You'll need to create a global lock object.

lock = threading.Lock()

Then around each access of the dictionary acquire and release the lock. The simplest way to do this is with the new(ish) with syntax.

with lock:
    dict[key] = value

answered Dec 13 '11 at 10:46

Andrew Wilkinson

10,682
3
35
38

i am restricted to the use of Python 2.4, would that be not available I guess ? – BuZz Dec 13 '11 at 11:09
No, the with is not available, but you can use lock.acquire() and lock.release instead(). – Andrew Wilkinson Dec 13 '11 at 13:35

score 10 · Answer 2 · edited May 23 '17 at 11:48

You may or may not need to use a lock, depending on how the Boolean is updated.

If the value of the Boolean doesn't depend on its previous value, then no lock is needed: writing and reading a Python dictionary is thread-safe by itself (except: writing while iterating is not allowed - but that's not allowed in single thread either). The memory visibility is similar to what would be achieved using volatile in some languages.

What's inherently not thread-safe is the "read-modify-write" -sequence, resulting in a race condition. If the value of the Boolean does depend on its previous value, then you have to use a lock, because otherwise thread A could first read the value, then thread B could change it, and then A would change it again, based on outdated value to start with.

score 8 · Accepted Answer · answered Dec 13 '11 at 12:01

8

If you need a lock (to avoid the race conditions Joonas described), and are stuck with Python 2.4,

import threading
lock = threading.Lock()

shared_dict = {}

def do_thing(user, value):
    lock.acquire()
    try:
        shared_dict[user] = value
    finally:
        # Always called, even if exception is raised in try block
        lock.release()

answered Dec 13 '11 at 12:01

dbr

165,801
69
278
343

let's say my writer wraps its modifications under that lock. Will the reader wait until this is released, i.e. writing has finished ? Or do I need to wrap the reads in that as well ? – BuZz Dec 13 '11 at 12:43
@jeromeG you'd probably need to have a similar locked function for reading (again just to avoid the "read-modify-write" race conditions) – dbr Dec 13 '11 at 23:06

score 2 · Answer 4 · answered Dec 13 '11 at 10:39

2

Use threading.LOCK.acquire() before updating the dictionary and use threading.LOCK.release(), once you are done updating it.

answered Dec 13 '11 at 10:39

Abhijit

62,056
18
131
204

1

This is strange, I would have expected the lock object to be specific to the dictionary, a little like a synchronized object/block/method in Java... I'm confused – BuZz Dec 13 '11 at 11:09

score -1 · Answer 5 · answered Dec 13 '13 at 13:05

-1

You don´t need to lock dictionary to this operation, because this operation is atomic, and GIL take care this for you. Unless you have operations like read-change-write, don't worry.

answered Dec 13 '13 at 13:05

Jan Palach

1

How would you make this Python Dictionary thread-safe?

5 Answers5

Linked