python3 accessing class attributes in a generator at initialisaton

Question

I'm currently switching a codebase over from python2 to python3 and come across an issue I don't quite understand :

class MyClass:
    var1 = True
    var2 = tuple([i for i in [1, 2,] if var1])

The above class was running quite happily in python2, but broke in python3.

I changed it to :

class MyClass:
    var1 = True
    var2 = tuple(i for i in [1, 2,] if var1)

because my understanding was the list comprehension was redundant, regardless it doesn't work. after some investigating it appears that list/tuple comprehensions behave in a way I don't quite understand when they're in the body of a class being initialized.

# Breaks in python 2 and 3
class MyClass:
    var1 = True
    var2 = tuple(i for i in [1, 2,] if var1)


# Works in python 2, breaks in 3
class MyClass:
    var1 = True
    var2 = [i for i in [1, 2,] if var1]

Any pointers on what's going on?

I don't believe that this question is quite a duplicate of the attached dup but I won't reopen it unless someone else do that. — Mazdak, May 08 '18 at 09:20
@Kasramvd Could you elaborate? I think Martijn's answer covers everything the OP could possibly want to know. — Aran-Fey, May 08 '18 at 09:48
@Aran-Fey Yes, that answer is comprehensive but I meant the question is not duplicate. The duplication is basically for the questions. Although one can argue that when you can find the answer in other not-duplicate question so what's the point not duplicating it. But, When you start a thread based on a particular root it will lead to its very own results and moreover, for questions like this, since there are many newer versions of Python since 3.1 and all of the have a lot of new features, one can suggest more updated answers. — Mazdak, May 08 '18 at 09:57
@Aran-Fey All in one, I mean that non trivial and good questions like this that are addressing some fundamental issues of the language always deserve to be discuss and reviewed. — Mazdak, May 08 '18 at 10:00
@Kasramvd I get it now, you're saying it's a different question because it asks about the difference between list comprehensions and generator expressions. I don't think that's a good enough reason to reopen the question though. It's just putting a different spin on the question *"Why does this list comprehension work in python2 but not in python3?"* - it's because list comprehensions don't have their own scope in python2, but generator expressions do. The answer to *"Why is a list comprehension different from a generator expression?"* is really just *"Because they're different expressions"*. — Aran-Fey, May 08 '18 at 10:47

Mazdak · Accepted Answer · 2018-05-08T09:24:46.003

Regarding the first case, what you have is actually passing a generator expression to a tuple function which is not even a class attribute. It's actually similar to the following code:

>>> class MyClass:
...     var1 = True
...     def func():
...         return var1
...     func()
... 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 5, in MyClass
  File "<stdin>", line 4, in func
NameError: global name 'var1' is not defined

Which as you can see again raises NameError in Python-2.7 and definitely 3.X. And the reason is that functions have their own namespace and without specifically declaring the state of that variable (global, local, nonlocal) or accessing then through instances of the class (self.var1) functions don't have access to outer bounded-namespaces.

List comprehensions, in other hand, in python-2 have access to the namespace of the object in which they're being defined. They are not like functions and don't have many features that function have. This is actually a two-sided relationship. This is to say that while you can use a variable inside list comprehension, what you define inside list comprehenion is also accessible outside it. And because people tend to use throwaway variables within list comprehensions this will cause your code to have variable leaks. But since Python-3.x list comprehension borrow the private namesspace from function so that they can have their own namespace.

interesting, thanks! is there a common pattern for doing what the currently broken code does, or is it a case of rethinking the logic entirely and moving it to `__init__` or similar? — ptr, May 08 '18 at 09:36
@ptr From an OOP perspective it's better to move your variable into a `__init__` which is easily accessible and you have more control over it. Unless, its cost is too much which its very unlikely in this case. — Mazdak, May 08 '18 at 09:49

python3 accessing class attributes in a generator at initialisaton

1 Answers1