Python decorator parameter scope

Question

I've implemented a retry decorator with some parameters:

def retry(max_tries: int = 3, delay_secs: float = 1, backoff: float = 1.5):
    print("level 1:", max_tries, delay_secs, backoff)
    def decorator(func):
        print("level 2:", max_tries, delay_secs, backoff)
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            nonlocal delay_secs  ## UnboundLocalError if remove this line
            print("level 3:", max_tries, delay_secs, backoff)

            for attempt in range(max_tries):
                try:
                    return func(*args, **kwargs)
                except Exception as e:
                    print(f"attempt {attempt} Exception: {e}  Sleeping {delay_secs}")
                    time.sleep(delay_secs)
                    delay_secs *= backoff
            print("exceeded maximun tries")
            raise e

        return wrapper

    return decorator

@retry(max_tries=4, delay_secs=1, backoff=1.25)
def something():
    raise Exception("foo")

something()

If I remove this line, I got UnboundLocalError

nonlocal delay_secs

But that only happens for delay_secs, NOT for max_tries or backoff! I tried reordering/renaming the params and still that delay param is problematic.

Can you help me understand why that parameter out of scope within the wrapper function but the other 2 parameters are just fine?

Python: 3.9.2

OS: Debian 11 Linux

IOW, it's because you do `delay_secs *= backoff`, Since that is an assignment statement to `delay_secs`. Essentially, for the same exact reasons as here: https://stackoverflow.com/questions/9264763/why-does-this-unboundlocalerror-occur-closure — juanpa.arrivillaga, May 17 '23 at 23:11

jsbueno · Accepted Answer · 2023-05-29T12:19:27.747

The answer is almost trivial: out of the 3 variables, delay_secs, max_tries and backoff, there is only one you write to in your inner function.

When one tries to retrieve the value of a variable for which there is no assignment inside a function, there is no ambiguity: Python automatically searches for that variable in outer scopes (functions where the inner function is nested in), then in the global, and finally the built-in scope. (If it is not found in any of those, a NameError is raised).

But the compiler of the Python runtime assumes that any variable that is assigned to in a function is a local variable - if one wants to change the values of variables in outer or global scopes, they have to be explicitly declared as so (with the nonlocal or global keywords, respectivelly).

So, since the value of delay_secs is changed, without the nonlocal declaration, Python assumes that is a local variable. When it tries to retrieve its value, both in your print call, and then in the implicit read when you use the *= operator (it has to retrieve the original value in order to multiply it), there is no local value assigned to it yet, so the error. Python "knows", because it marks it at compile time, that at some point that function should have a local variable named delay_secs - so you don't get a NameError - but since there is no value to be read, it raises UnboundLocalError instead.

Note that some other languages would use the outer scope variable for reading, until it was assigned in the inner function, at which point a local variable would be created - this is not how Python works, and the rationale for that is written in the PEP which created the nonlocal keyword itself: https://peps.python.org/pep-3104/

Python decorator parameter scope

1 Answers1