Scope error when returning function from function

Question

I am having issues with using function generators in python, i.e. returning functions from a function, where some of the inputs are defined via the outer function's arguments, yielding a function with lower arity. I have used this method before in other cases without issue but I have run into an error in this case and haven't been able to understand the reason. I have the following function that produces a function:

def func1(s: set, f: Callable) -> Callable:
    def func2(x: int) -> dict:
        while x:
            s = s - {x}
            x = f(x)
        return s
    return func2

f2 = func1({1,2,3,4}, lambda x: x-1)

If I try f2(3), I get the error "local variable 's' referenced before assignment" related to the s = s - {x} statement. I don't understand why s is not part of the scope since I assumed it was fixed when func1 was called to create f2.

I was able to fix this issue by instead using this method: I combine all the arguments into one function, say func3 and use the functools.partial operation, which behaves as expected:

from functools import partial
def func3(x: int, s: set, f: Callable) -> dict:
    while x:
        s = s - {x}
        x = f(x)
    return s

f2 = partial(func3, s={1,2,3,4}, f=lambda x: x-1)

When I call f2(3) I get {4} as expected.

My understanding is that both of these approaches should yield the same result; however, I am apparently missing something about how variable scope operates in the first case. Any insight would be greatly appreciated.

Answer 1

The problem with your first piece of code is that s is a local variable inside func2. It has no connection to the variable s that you've defined inside func1. Remember that any variable that is assigned to within a function is local, unless you declare it otherwise.

The fix is to add nonlocal s to your definition of func2. Python will then realize that the s referred to inside func2 is the same as the s inside func1.

@KellyBundy points out that this may still not give you exactly what you want. If you simply make s nonlocal, you are effectively saying that these two variables point point to the same set. Any changes made to s are "permanent". The second time f2 is called, the value of s will be whatever it was at the end of the first time f2 is called. It is uncertain if this is what is intended.

If you want s to be reset each time f2 is called, you need to keep the two ss separate. The simplest way would be to start off func2 with s0 = s, and then just use s0 within func2. Since sis never modified withinfunc2, it refers to the outer variable. And you have a different variable s0withinfunc2` that can be modified as you wish.