Why is __slots__ behaving differently in Python 2 and 3 when inheriting from an abstract base class

Question

I created the following class to store changeable points on a plane in a memory-efficient manner - I need a mutable equivalent of namedtuple('Point', 'x y'). Since instance dictionaries are big, I thought I'd go for __slots__:

from collections import Sequence

class Point(Sequence):
    __slots__ = ('x', 'y')

    def __init__(self, x=0, y=0):
        self.x = x
        self.y = y

    def __getitem__(self, item):
        return getattr(self, self.__slots__[item])

    def __setitem__(self, item, value):
        return setattr(self, self.__slots__[item], value)

    def __repr__(self):
        return 'Point(x=%r, y=%r)' % (self.x, self.y)

    def __len__(self):
        return 2

When testing it on Python 3, everything seemed to be OK:

>>> pt = Point(12, 42)
>>> pt[0], pt.y
(12, 42)
>>> pt.x = 5
>>> pt
Point(x=5, y=42)
>>> pt.z = 6
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Point' object has no attribute 'z'

However on Python 2, I can set the attribute z even when it is not in slots:

>>> pt = Point(12, 42)
>>> pt.z = 5
>>> pt.z
5
>>> pt.__slots__
('x', 'y')
>>> pt.__dict__
{'z': 5}

Why is that so, and why the difference between Python 2 and Python 3?

Answer 1

The Python 2 data model says the following on __slots__:

• When inheriting from a class without __slots__, the __dict__ attribute of that class will always be accessible, so a __slots__ definition in the subclass is meaningless.

And that is what is happening here. In Python 2 the abstract base classes in the collections module did not have the __slots__ at all:

>>> from collections import Sequence
>>> Sequence.__slots__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: type object 'Sequence' has no attribute '__slots__'

This was reported as the issue 11333 in CPython issue tracker, and was fixed in Python 3.3.

In Python 3.3+ the Sequence abstract base class now has __slots__ set to an empty tuple:

>>> from collections import Sequence
>>> Sequence.__slots__
()

---

Thus in Python 2, you cannot inherit from a collections base class and have memory efficient storage with __slots__ at the same time.

---

Note however that even though the documentation on collections abstract base classes claims that

These ABCs allow us to ask classes or instances if they provide particular functionality, for example:

size = None
if isinstance(myvar, collections.Sized):
    size = len(myvar)

This is not the case with Sequence; simply implementing all the methods required by the Sequence does not make instances of your class to pass the isinstance check.

The reason for that is that the Sequence class does not have a __subclasshook__; and in its absence, the parent class __subclasshook__ is consulted instead; in this case Sized.__subclasshook__; and that returns NotImplemented if the tested-against class wasn't exactly Sized.

On the other hand, one couldn't distinguish between a mapping type and a sequence type by the magic methods, as both of them can have the exactly same magic methods - of collections.OrderedDict has all the magic methods of a Sequence, including the __reversed__ method, yet it isn't a sequence.

However, you still do not need to inherit from Sequence to make isinstance(Point, Sequence) return True. In the following example, the Point is the same, except derived from object instead of Sequence, on Python 2:

>>> pt = Point(12, 42)
>>> pt.z = 5
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Point' object has no attribute 'z'
>>> isinstance(pt, Sequence)
False
>>> Sequence.register(pt)
>>> isinstance(pt, Sequence)
True

You can register any class as a subclass of the abstract base class for the purpose of isinstance checks; and of the extra mix-in methods, you really need to implement only count and index; the functionality for others will be filled in by Python runtime.