93

The following code spits out 1 twice, but I expect to see 0 and then 1.

def pv(v) :
  print v

x = []
for v in range(2):
  x.append(lambda : pv(v))

for xx in x:
  xx()

I expected python lambdas to bind to the reference a local variable is pointing to, behind the scenes. However that does not seem to be the case. I have encountered this problem in a large system where the lambda is doing modern C++'s equivalent of a bind ('boost::bind' for example) where in such case you would bind to a smart ptr or copy construct a copy for the lambda.

So, how do I bind a local variable to a lambda function and have it retain the correct reference when used? I'm quite gobsmacked with the behaviour since I would not expect this from a language with a garbage collector.

wjandrea
  • 28,235
  • 9
  • 60
  • 81
Hassan Syed
  • 20,075
  • 11
  • 87
  • 171
  • 3
    I've seen several variants of this question. someone needs to aggregate all of them, change the title to something memorable, and then tell the internet to google that – Shep May 04 '12 at 16:43
  • 2
    ah, here you go, near duplicate: [lexical-closures-in-python](http://stackoverflow.com/q/233673/915501) – Shep May 04 '12 at 16:48
  • @shep, yes , I couldn't phrase it correctly, I was trying to replicate the way lua works. – Hassan Syed May 08 '12 at 20:52
  • 3
    The strange part of Python is that loop variables persist after the loop, so that functions defined inside the loop and referencing the loop variable don't throw a `NameError` if called outside the loop. In most other languages, loop variables are only accessible within the loop itself. – BallpointBen May 14 '18 at 20:19
  • "I'm quite gobsmacked with the behaviour since I would not expect this from a language with a garbage collector." I can't understand this viewpoint. Why would *garbage collection*, of all things, have anything to do with early vs. late binding? – Karl Knechtel Aug 18 '22 at 04:09

2 Answers2

146

Change x.append(lambda : pv(v)) to x.append(lambda v=v: pv(v)).

You expect "python lambdas to bind to the reference a local variable is pointing to, behind the scene", but that is not how Python works. Python looks up the variable name at the time the function is called, not when it is created. Using a default argument works because default arguments are evaluated when the function is created, not when it is called.

This is not something special about lambdas. Consider:

x = "before foo defined"
def foo():
    print x
x = "after foo was defined"
foo()

prints

after foo was defined
Steven Rumbalski
  • 44,786
  • 9
  • 89
  • 119
43

The lambda's closure holds a reference to the variable being used, not its value, so if the value of the variable later changes, the value in the closure also changes. That is, the closure variable's value is resolved when the function is called, not when it is created. (Python's behavior here is not unusual in the functional programming world, for what it's worth.)

There are two solutions:

  1. Use a default argument, binding the current value of the variable to a local name at the time of definition. lambda v=v: pv(v)

  2. Use a double-lambda and immediately call the first one. (lambda v: lambda: pv(v))(v)

kindall
  • 178,883
  • 35
  • 278
  • 309
  • I like the second approach, as the variable I want to freeze at its current value is not necessarily a parameter of the lambda. – g.a Nov 26 '20 at 18:12
  • 1
    @g.a. the value doesn't need to already be a parameter of the lambda for the first technique to work. the opposite, really – kindall Nov 28 '20 at 05:54
  • What do you mean by calling lambda a closure? Functions contain references only to the `global` namespace (https://docs.python.org/3/library/types.html#standard-interpreter-types) (also to default arguments, but this is not the case, https://docs.python.org/3/reference/datamodel.html#the-standard-type-hierarchy). – Yaroslav Nikitenko Jun 19 '22 at 16:06
  • Are you asking me to define "closure" or are you under the impression that Python functions don't close over variables from the enclosing scope? They certainly do. https://til.hashrocket.com/posts/ykhyhplxjh-examining-the-closure – kindall Jun 20 '22 at 16:26