How do I capture the CURRENT values of closure variables immutably in Python?

Question

If I do

def f():
    a = 1
    g = lambda: a
    a = 2
    return g
print(f()())

The value that is printed is 2, because a is mutated after g is constructed.
How do I get g to capture the value of a statically so that later modifications are ignored?

score 2 · Accepted Answer · answered May 29 '17 at 10:42

For simple cases, such as when the code is short and we don't have many variables to capture, we can create a temporary lambda and call it:

def f():
    a = 1
    g = (lambda a: lambda: a)(a)
    a = 2
    return g

The issue here is that the code can quickly become harder to read.

Alternatively, we can capture the variable as an optional argument:

def f():
    a = 1
    g = lambda a=a: a
    a = 2
    return g

The issue here is, of course, that we might not want the caller to be able to specify this parameter.
(And the code can be a little less readable, too.)

A fully general solution might be the following, except that it does not capture globals:

def bind(*args, **kwargs):
    # Use '*args' so that callers aren't limited in what names they can specify
    func               = args[0]
    include_by_default = args[1] if len(args) > 1 else None
    # if include_by_default == False, variables are NOT bound by default
    if include_by_default == None: include_by_default = not kwargs
    fc = func.__code__
    fv = fc.co_freevars
    q = func.__closure__
    if q:
        ql = []
        i = 0
        for qi in q:
            fvi = fv[i]
            ql.append((lambda v: (lambda: v).__closure__[0])(
                kwargs.get(fvi, qi.cell_contents))
                if include_by_default or fvi in kwargs
                else qi)
            i += 1
        q = q.__class__(ql)
        del ql
    return func.__class__(fc, func.__globals__, func.__name__, func.__defaults__, q)

The reason I do not attempt to capture globals here is that the semantics can get confusing -- if an inner function says global x; x = 1, it certainly does want the global x to be modified, so suppressing this change would quickly make the code very counterintuitive.

However, barring that, we would be able to simply use it as follows:

def f():
    a = 1
    g = bind(lambda: a)
    a = 2
    return g
print(f()())

And voilà, a is instantly captured. And if we want to only capture some variables, we can do:

def f():
    a = 1
    b = 2
    g = bind(lambda: a + b, b=5)  # capture 'b' as 5; leave 'a' free
    a = 2
    b = 3
    return g
print(f()())

score 1 · Answer 2 · answered Aug 04 '20 at 22:45

In python3.8 the CellType class was added to types, which means you can create a function with a custom closure. That allows us to write a function that converts functions with closures that reference a parent frame to closures with static values:

from types import FunctionType, CellType

def capture(f):
    """ Returns a copy of the given function with its closure values and globals shallow-copied """
    closure = tuple(CellType(cell.cell_contents) for cell in f.__closure__)
    return FunctionType(f.__code__, f.__globals__.copy(), f.__name__, f.__defaults__, closure)

print([f() for f in [        lambda: i  for i in range(5)]]) # Outputs [4, 4, 4, 4, 4]
print([f() for f in [capture(lambda: i) for i in range(5)]]) # Outputs [0, 1, 2, 3, 4]

The capture function could be tweaked in a few ways; The current implementation captures all globals and free vars, and does so shallowly. You could decide to deepcopy the captured values, to capture only the free vars, to capture only specific variable names according to an argument, etc.

How do I capture the CURRENT values of closure variables immutably in Python?

2 Answers2