Creating functions (or lambdas) in a loop (or comprehension)

Question

I'm trying to create functions inside of a loop:

functions = []

for i in range(3):
    def f():
        return i

    # alternatively: f = lambda: i

    functions.append(f)

The problem is that all functions end up being the same. Instead of returning 0, 1, and 2, all three functions return 2:

print([f() for f in functions])
# expected output: [0, 1, 2]
# actual output:   [2, 2, 2]

Why is this happening, and what should I do to get 3 different functions that output 0, 1, and 2 respectively?

Answer 1

You're running into a problem with late binding -- each function looks up i as late as possible (thus, when called after the end of the loop, i will be set to 2).

Easily fixed by forcing early binding: change def f(): to def f(i=i): like this:

def f(i=i):
    return i

Default values (the right-hand i in i=i is a default value for argument name i, which is the left-hand i in i=i) are looked up at def time, not at call time, so essentially they're a way to specifically looking for early binding.

If you're worried about f getting an extra argument (and thus potentially being called erroneously), there's a more sophisticated way which involved using a closure as a "function factory":

def make_f(i):
    def f():
        return i
    return f

and in your loop use f = make_f(i) instead of the def statement.

Answer 2

The Explanation

The issue here is that the value of i is not saved when the function f is created. Rather, f looks up the value of i when it is called.

If you think about it, this behavior makes perfect sense. In fact, it's the only reasonable way functions can work. Imagine you have a function that accesses a global variable, like this:

global_var = 'foo'

def my_function():
    print(global_var)

global_var = 'bar'
my_function()

When you read this code, you would - of course - expect it to print "bar", not "foo", because the value of global_var has changed after the function was declared. The same thing is happening in your own code: By the time you call f, the value of i has changed and been set to 2.

The Solution

There are actually many ways to solve this problem. Here are a few options:

• Force early binding of i by using it as a default argument

  Unlike closure variables (like i), default arguments are evaluated immediately when the function is defined:

  for i in range(3):
      def f(i=i):  # <- right here is the important bit
          return i
  
      functions.append(f)
  

  To give a little bit of insight into how/why this works: A function's default arguments are stored as an attribute of the function; thus the current value of i is snapshotted and saved.

  >>> i = 0
  >>> def f(i=i):
  ...     pass
  >>> f.__defaults__  # this is where the current value of i is stored
  (0,)
  >>> # assigning a new value to i has no effect on the function's default arguments
  >>> i = 5
  >>> f.__defaults__
  (0,)
  

• Use a function factory to capture the current value of i in a closure

  The root of your problem is that i is a variable that can change. We can work around this problem by creating another variable that is guaranteed to never change - and the easiest way to do this is a closure:

  def f_factory(i):
      def f():
          return i  # i is now a *local* variable of f_factory and can't ever change
      return f
  
  for i in range(3):           
      f = f_factory(i)
      functions.append(f)
  

• Use functools.partial to bind the current value of i to f

  functools.partial lets you attach arguments to an existing function. In a way, it too is a kind of function factory.

  import functools
  
  def f(i):
      return i
  
  for i in range(3):    
      f_with_i = functools.partial(f, i)  # important: use a different variable than "f"
      functions.append(f_with_i)
  

Caveat: These solutions only work if you assign a new value to the variable. If you modify the object stored in the variable, you'll experience the same problem again:

>>> i = []  # instead of an int, i is now a *mutable* object
>>> def f(i=i):
...     print('i =', i)
...
>>> i.append(5)  # instead of *assigning* a new value to i, we're *mutating* it
>>> f()
i = [5]

Notice how i still changed even though we turned it into a default argument! If your code mutates i, then you must bind a copy of i to your function, like so:

• def f(i=i.copy()):
• f = f_factory(i.copy())
• f_with_i = functools.partial(f, i.copy())

Answer 3

For those coming to this question using lambda:

The solution is simply to replace lambda: i by lambda i=i: i.

functions = []
for i in range(3): 
    functions.append(lambda i=i: i)
print([f() for f in functions])
# [0, 1, 2]

Example use-case: How to have a lambda function evaluate a variable now (and not postponed)

Answer 4

To add onto @Aran-Fey's excellent answer, in the second solution you might also wish to modify the variable inside your function which can be accomplished with the keyword nonlocal:

def f_factory(i):
    def f(offset):
      nonlocal i
      i += offset
      return i  # i is now a *local* variable of f_factory and can't ever change
    return f

for i in range(3):           
    f = f_factory(i)
    print(f(10))

Answer 5

You can try like this:

l=[]
for t in range(10):
    def up(y):
        print(y)
    l.append(up)
l[5]('printing in 5th function')

Answer 6

You have to save the each of the i value in a separate space in memory e.g.:

class StaticValue:
    val = None

    def __init__(self, value: int):
        StaticValue.val = value

    @staticmethod
    def get_lambda():
        return lambda x: x*StaticValue.val


class NotStaticValue:
    def __init__(self, value: int):
        self.val = value

    def get_lambda(self):
        return lambda x: x*self.val


if __name__ == '__main__':
    def foo():
        return [lambda x: x*i for i in range(4)]

    def bar():
        return [StaticValue(i).get_lambda() for i in range(4)]

    def foo_repaired():
        return [NotStaticValue(i).get_lambda() for i in range(4)]

    print([x(2) for x in foo()])
    print([x(2) for x in bar()])
    print([x(2) for x in foo_repaired()])

Result:
[6, 6, 6, 6]
[6, 6, 6, 6]
[0, 2, 4, 6]

Answer 7

just modify the last line for

functions.append(f())

Edit: This is because f is a function - python treats functions as first-class citizens and you can pass them around in variables to be called later on. So what your original code is doing is appending the function itself to the list, while what you want to do is append the results of the function to the list, which is what the line above achieves by calling the function.