Python: using lambda as threading target causes strange behavior

Question

I am experiencing a strange behavior in Python. When I used a lambda as a target of a thread, the behavior was inconsistent.

The first example is like this:

import time,threading

locker= threading.RLock()

def func(obj):
  while obj['count']>0:
    with locker: print 'thread',obj,id(obj)
    obj['count']-= 1
    time.sleep(0.1)
  with locker: print 'finished',obj,id(obj)

def make_thread1():
  threads= []
  objs= {}
  for i in range(2):
    objs[i]= {}
    objs[i]['id']= i
    objs[i]['count']= (i+2)*2
    t= threading.Thread(name='func'+str(i), target=lambda: func(objs[i]))
    t.start()
    threads.append(t)
  return threads,objs

if __name__=='__main__':
  threads,objs= make_thread1()
  for t in threads:
    t.join()

There were two patterns of results. One was

thread {'count': 4, 'id': 0} 139911658041704
thread {'count': 6, 'id': 1} 139911658041984
thread {'count': 3, 'id': 0} 139911658041704
thread {'count': 5, 'id': 1} 139911658041984
thread {'count': 4, 'id': 1} 139911658041984
thread {'count': 2, 'id': 0} 139911658041704
thread {'count': 3, 'id': 1} 139911658041984
thread {'count': 1, 'id': 0} 139911658041704
thread {'count': 2, 'id': 1} 139911658041984
finished {'count': 0, 'id': 0} 139911658041704
thread {'count': 1, 'id': 1} 139911658041984
finished {'count': 0, 'id': 1} 139911658041984

This is a result I expected. However when running this code several times, sometimes it resulted like this:

thread {'count': 6, 'id': 1} 140389870428800
thread {'count': 5, 'id': 1} 140389870428800
thread {'count': 4, 'id': 1} 140389870428800
thread {'count': 3, 'id': 1} 140389870428800
thread {'count': 2, 'id': 1} 140389870428800
thread {'count': 1, 'id': 1} 140389870428800
finished {'count': 0, 'id': 1} 140389870428800
finished {'count': 0, 'id': 1} 140389870428800

When making the threads, lambda:func(objs[0]) and lambda:func(objs[1]) were defined as the target functions respectively, but actually both target functions were lambda:func(objs[1]) (but different instances).

I cannot understand why this happened.

Well, one possibility would be that I was using a local variable i in making the lambda functions. But it should be evaluated when t.start() was executed...? Then why there were two patterns of the results?

For more investigation, I modified the code without lambda:

class TObj:
  def __init__(self):
    self.objs= None
  def f(self):
    func(self.objs)

def make_thread2():
  threads= []
  classes= {}
  for i in range(2):
    classes[i]= TObj()
    classes[i].objs= {}
    classes[i].objs['id']= i
    classes[i].objs['count']= (i+2)*2
    t= threading.Thread(name='func'+str(i), target=classes[i].f)
    t.start()
    threads.append(t)
  return threads,classes

if __name__=='__main__':
  threads,classes= make_thread2()
  for t in threads:
    t.join()

This code worked perfectly:

thread {'count': 4, 'id': 0} 140522771444352
thread {'count': 6, 'id': 1} 140522771445472
thread {'count': 3, 'id': 0} 140522771444352
thread {'count': 5, 'id': 1} 140522771445472
thread {'count': 2, 'id': 0} 140522771444352
thread {'count': 4, 'id': 1} 140522771445472
thread {'count': 1, 'id': 0} 140522771444352
thread {'count': 3, 'id': 1} 140522771445472
finished {'count': 0, 'id': 0} 140522771444352
thread {'count': 2, 'id': 1} 140522771445472
thread {'count': 1, 'id': 1} 140522771445472
finished {'count': 0, 'id': 1} 140522771445472

I want to understand the reasons why the inconsistency of the first code happened.

I also want to know how to make lambda functions in loops safely. If the above problem was caused by using a loop variable i in a lambda function, we should avoid to generate lambda functions in loops. It would be inconvenient; I could code like the latter one, but it was longer. Are there any good ideas?

score 1 · Accepted Answer · edited May 23 '17 at 12:09

An idea to use lambda with loop variables is described in Deferred evaluation with lambda in Python

We can use loop (local) variables in lambda by using default argument values. In my case, making a lambda function like:

t= threading.Thread(name='func'+str(i), target=lambda i=i: func(objs[i]))

This worked as same as the second solution where a class object is used.

About the other point: when Thread.start evaluates the target? It would depend on the system. It could vary. If there was a delay, it would be evaluated after i was incremented. This would be a reason of the inconsistency.

Python: using lambda as threading target causes strange behavior

1 Answers1