Python closure function losing outer variable access

Question

I just learned python @ decorator, it's cool, but soon I found my modified code coming out weird problems.

def with_wrapper(param1):
    def dummy_wrapper(fn):
        print param1
        param1 = 'new'
        fn(param1)
    return dummy_wrapper

def dummy():
    @with_wrapper('param1')
    def implementation(param2):
        print param2

dummy()

I debug it, it throws out exception at print param1

UnboundLocalError: local variable 'param1' referenced before assignment

If I remove param1 = 'new' this line, without any modify operation (link to new object) on variables from outer scope, this routine might working.

Is it meaning I only have made one copy of outer scope variables, then make modification?

Thanks Delnan, it's essential to closure. Likely answer from here: What limitations have closures in Python compared to language X closures?

Similar code as:

def e(a):
    def f():
        print a
        a = '1'
    f()
e('2')

And also this seems previous annoying global variable:

a = '1'
def b():
    #global a
    print a
    a = '2'
b()

This is fixed by add global symbol. But for closure, no such symbol found. Thanks unutbu, Python 3 gave us nonlocal.

I know from above directly accessing to outer variable is read-only. but it's kind of uncomfortable to see preceded reading variable(print var) is also affected.

possible duplicate of [What limitations have closures in Python compared to language X closures?](http://stackoverflow.com/questions/141642/what-limitations-have-closures-in-python-compared-to-language-x-closures) — , Aug 29 '12 at 16:10
Yes, this happens in closures. like this: def e(a): def f(): print a a = '1' f() e('123') — V.E.O, Aug 30 '12 at 02:03
Why do you even need to assign to the same name? Why not `param3 = new` ... `fn(param3)`? — detly, Aug 30 '12 at 02:25
Hi detly, it's just demo to print and assign one parameter. if this var is from any outer scope, problems works still — V.E.O, Aug 30 '12 at 02:32

unutbu · Accepted Answer · 2012-08-29T16:17:41.993

22

When Python parses a function, it notes whenever it finds a variable used on the left-hand side of an assignment, such as

param1 = 'new'

It assumes that all such variables are local to the function. So when you precede this assignment with

print param1

an error occurs because Python does not have a value for this local variable at the time the print statement is executed.

In Python3 you can fix this by declaring that param1 is nonlocal:

def with_wrapper(param1):
    def dummy_wrapper(fn):
        nonlocal param1
        print param1
        param1 = 'new'
        fn(param1)
    return dummy_wrapper

In Python2 you have to resort to a trick, such as passing param1 inside a list (or some other mutable object):

def with_wrapper(param1_list):
    def dummy_wrapper(fn):
        print param1_list[0]
        param1_list[0] = 'new'   # mutate the value inside the list
        fn(param1_list[0])
    return dummy_wrapper

def dummy():
    @with_wrapper(['param1'])   # <--- Note we pass a list here
    def implementation(param2):
        print param2

edited Aug 29 '12 at 16:17

answered Aug 29 '12 at 16:11

unutbu

842,883
184
1,785
1,677

1

... put the `print` statement after the assignment? – kindall Aug 29 '12 at 16:13
Thanks for explaining. But I have no idea about why param1 = 'new' reassign the outer param but to make one local var. – V.E.O Aug 30 '12 at 02:08
3

Perhaps you and I as humans would prefer Python to check the outer scopes first for a `param1` variable, and if it exists, use that, and only if it does not exist, then assume it is a local variable. But that would force Python to wait until runtime to decide what is a local variable of the function. I think this would slow down Python's performance significantly. Every time the function were called, localness would have to be redetermined. So instead, Python uses assignments to recognize local variables *at parse-time*, when the def block is executed, rather than when the function is called. – unutbu Aug 30 '12 at 03:08
@unutbu Thanks, now I thinks this optimization method is acceptable. – V.E.O Aug 30 '12 at 03:25
@unutbu nope, it would NOT need to be dynamically determined on runtime, both Python and JavaScript have lexical scoping for normal variables (`self` in Python or `this` in JavaScript are completely different beasts), it was just a historical design decision, explained in https://www.python.org/dev/peps/pep-3104 – Aprillion Jun 07 '17 at 13:25

score 0 · Answer 2 · answered Aug 29 '12 at 16:12

0

You assign param1 in the function, which makes param1 a local variable. However, it hasn't been assigned at the point you're printing it, so you get an error. Python doesn't fall back to looking for variables in outer scopes.

answered Aug 29 '12 at 16:12

kindall

178,883
35
278
309

_print param1_ is one example, in real code, any reference upon param1 is prohibited when param1 = xx exists – V.E.O Aug 30 '12 at 01:56

DavidJ · Answer 3 · 2018-05-03T19:07:05.407

Handy for temporary situations or exploratory code (but otherwise not good practice):

If you're wanting to capture a variable from the outer scope in Python 2.x then using global is also an option (with the usual provisos).

While the following will throw (assignment of outer1 within inner makes it local and hence unbounded in the if condition):

def outer():
    outer1 = 1
    def inner():
        if outer1 == 1:
            outer1 = 2
            print('attempted to accessed outer %d' % outer1)

This will not:

def outer():
    global outer1
    outer1 = 1
    def inner():
        global outer1
        if outer1 == 1:
            outer1 = 2
            print('accessed outer %d' % outer1)

Python closure function losing outer variable access

3 Answers3