About python closure

Question

Below is an example I got from someone's blog about python closure. I run it in python 2.7 and get a output different from my expect.

flist = []

for i in xrange(3):
    def func(x):
        return x*i
    flist.append(func)

for f in flist:
    print f(2)

My expected output is: 0, 2, 4
But the output is: 4, 4, 4
Is there anyone could help to explain it?
Thank you in advance.

Walter Mundt · Accepted Answer · 2012-07-10T18:13:30.640

17

Loops do not introduce scope in Python, so all three functions close over the same i variable, and will refer to its final value after the loop finishes, which is 2.

It seems as though nearly everyone I talk to who uses closures in Python has been bitten by this. The corollary is that the outer function can change i but the inner function cannot (since that would make i a local instead of a closure based on Python's syntactic rules).

There are two ways to address this:

# avoid closures and use default args which copy on function definition
for i in xrange(3):
    def func(x, i=i):
        return x*i
    flist.append(func)

# or introduce an extra scope to close the value you want to keep around:
for i in xrange(3):
    def makefunc(i):
        def func(x):
            return x*i
        return func
    flist.append(makefunc(i))

# the second can be simplified to use a single makefunc():
def makefunc(i):
    def func(x):
        return x*i
    return func
for i in xrange(3):
    flist.append(makefunc(i))

# if your inner function is simple enough, lambda works as well for either option:
for i in xrange(3):
    flist.append(lambda x, i=i: x*i)

def makefunc(i):
    return lambda x: x*i
for i in xrange(3):
    flist.append(makefunc(i))

edited Jul 10 '12 at 18:13

answered Jul 10 '12 at 07:33

Walter Mundt

24,753
5
53
61

Note for other readers: Python 3 adds the `nonlocal` keyword, which would allow each `func` to change the value of `i`, which in turn would affect the others. Not useful in this case, but potentially handy if you had several inner functions. – Walter Mundt Jul 10 '12 at 09:34
You could simplify the last one a bit more with lambda, though that restricts what func() can do. – Dubslow Jul 10 '12 at 09:36
@Dubslow I don't think that would simplify it. def looks a lot nicer imo – jamylak Jul 10 '12 at 11:49
Added the `lambda`-based options, since they do make sense sometimes for small things. It's true that I rarely end up needing them these days; in 2.7 generator/dict/set comprehensions have eaten up a lot of their usefulness. – Walter Mundt Jul 10 '12 at 18:14

score 4 · Answer 2 · answered Jul 10 '12 at 07:31

4

You are not creating closures. You are generating a list of functions which each access the global variable i which is equal to 2 after the first loop. Thus you end up with 2 * 2 for each function call.

answered Jul 10 '12 at 07:31

mhawke

84,695
9
117
138

Thank you for the answer. If I want to create closure to get my expected output, how to change the code? Could you please give me some suggestion? – Alex.Zhang Jul 10 '12 at 07:34
@Alex.Zhang: Well, you did ask for an explanation for why the behaviour experienced is not the same as you expected. See http://stackoverflow.com/a/11408601/21945 for a solution. – mhawke Jul 10 '12 at 07:38

score 1 · Answer 3 · answered Jul 10 '12 at 07:35

1

Each function accesses the global i.

functools.partial comes to rescue:

from functools import partial
flist = []

for i in xrange(3):
    def func(x, multiplier=None):
        return x * multiplier
    flist.append(partial(func, multiplier=i))

answered Jul 10 '12 at 07:35

Matthias

12,873
6
42
48

About python closure

3 Answers3