13

I have a following simple code:

def get():
    return [lambda: i for i in [1, 2, 3]]

for f in get():
    print(f())

As expected from my python knowledge, output is 3 - entire list will contain last value of i. But how this works internally?

AFAIK, python variables are simply reference to objects, so first closure must enclose object first i reference - and this object is definitely 1, not 3 O_O. How it happens that python closure encloses variable itself instead of object this variable reference? Does it save variable name as plain text, some "reference to variable" or what?

Gareth Latty
  • 86,389
  • 17
  • 178
  • 183
grigoryvp
  • 40,413
  • 64
  • 174
  • 277

3 Answers3

13

As @thg435 points out, a lambda will not encapsulate the values at that moment, but rather the scope. There are too small ways you can address this:

lambda default argument "hack"

[ lambda v=i: v for i in [ 1, 2, 3 ] ]

Or use functools.partial

from functools import partial
[ partial(lambda v: v, i) for i in [ 1, 2, 3 ] ]

Essentially you have to move the scope to be local to the function you are creating. Generally I like using partial more often since you can pass it a callable, and any args and kargs to create a callable with a proper closure. Internally, it is wrapping your original callable so the scope is shifted for you.

jdi
  • 90,542
  • 19
  • 167
  • 203
  • 5
    +1 For using partial here, I think it's a much cleaner approach. You might want to edit your code though, spaces inside list brackets are bad according to PEP-8. (I know you were following the asker, I edited it there). – Gareth Latty Jun 19 '12 at 21:47
  • @Lattyware: Thanks! lambdas are cool but I always find them more messy to read. partial just feels so much more readable and portable. – jdi Jun 19 '12 at 21:48
  • 100% agree, I try to avoid lambdas wherever I can, and this is a prime case where partial is both more clear and simpler. – Gareth Latty Jun 19 '12 at 21:49
10

Closures don't refer to variables but rather to scopes. Since the last value of i in its scope is '3', all three closures return the same. To "lock" the current value of a variable, create a new scope just for it:

def get() : return [ (lambda x: lambda: x)(i) for i in [ 1, 2, 3 ] ]
for f in get() : print( f() )
georg
  • 211,518
  • 52
  • 313
  • 390
  • Is it some additional information available on the subject so i can see what hidden variables are used to "save" scope, access it, etc? – grigoryvp Jun 19 '12 at 21:44
  • 2
    `(lambda x: lambda : x)(i)` is one of the ugliest things I've seen done in Python. Ick. (Not that your answer is wrong or bad, per-se, just saying - it's hard to read). – Gareth Latty Jun 19 '12 at 21:49
  • 3
    @EyeofHell: I think [pep-227](http://www.python.org/dev/peps/pep-0227/) is the canonical document about python scoping rules. Also, there are some good answers on this here on SO, e.g. [here](http://stackoverflow.com/questions/291978/short-description-of-python-scoping-rules) – georg Jun 19 '12 at 21:50
  • @Lattyware: de gustibus non est disputandum. – georg Jun 19 '12 at 21:52
  • @thg435 This is exactly why I added my little disclaimer ;). – Gareth Latty Jun 19 '12 at 21:54
  • @Lattyware, it's hard to read because Python was never designed from the bottom-up to elegantly support these sorts of functional-style constructs. The equivalent code in a language like, e.g., Scheme would be both more idiomatic and more aesthetically pleasing. – Greg E. Jun 19 '12 at 21:55
  • 2
    @GregE. I don't think it's impossible to do nicely in Python. I think jdi's answer of `functools.partial()` and `lambda` mixed is the nicest solution, it describes what is being done in a nicer way - from my point of view. – Gareth Latty Jun 19 '12 at 21:56
  • @Lattyware, it certainly is nicer, and I'm not saying that it's impossible to do in a reasonably clear manner. Nevertheless, the fact that it requires importing a library and isn't cleanly achievable using core language constructs suggests that it's never been a major design priority. I like Python a lot, but it's always been dismissive of, if not actively hostile toward, language features that are integral to functional style programming. – Greg E. Jun 19 '12 at 22:02
  • @EyeofHell: BTW I might be not completely correct about how this works. I posted a [follow up question](http://stackoverflow.com/questions/11110304/auto-bind-scope-variables-in-a-closure) in hope that someone comes up with a better explanation. – georg Jun 19 '12 at 22:20
4

Each lambda is actually referring to the same i, which is a variable created by the list comprehension. Upon termination of the list comprehension, i maintains the value of the final element that it was assigned to until it goes out of scope (which is prevented by encapsulating it within a function and returning it, namely the lambda). As others have pointed out, closures don't maintain copies of values, but rather maintain references to variables that were defined within their scope.

Greg E.
  • 2,722
  • 1
  • 16
  • 22