17

I've heard that __slots__ makes objects faster by avoiding a dictionary lookup. My confusion comes from Python being a dynamic language. In a static language, we avoid a dictionary lookup for a.test by doing a compile-time optimisation to save the index in the instruction we run.

Now, in Python, a could just as easily be another object that has a dictionary or a different set of attributes. It seems like we'll still have to do a dictionary lookup - the only difference seems to be that we only need one dictionary for the class, rather than a dictionary for each object.

With this rational,

  1. How does __slots__ avoid a dictionary lookup?
  2. Does slots make accessing objects faster?
Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
Casebash
  • 114,675
  • 90
  • 247
  • 350

2 Answers2

21

__slots__ does not (significantly) speed up attribute access:

>>> class Foo(object):
...     __slots__ = ('spam',)
...     def __init__(self):
...         self.spam = 'eggs'
... 
>>> class Bar(object):
...     def __init__(self):
...         self.spam = 'eggs'
... 
>>> import timeit
>>> timeit.timeit('t.spam', 'from __main__ import Foo; t=Foo()')
0.07030296325683594
>>> timeit.timeit('t.spam', 'from __main__ import Bar; t=Bar()')
0.07646608352661133

The goal of using __slots__ is to save memory; instead of using a .__dict__ mapping on the instance, the class has descriptors objects for each and every attribute named in __slots__ and instances have the attribute assigned wether or not they have an actual value:

>>> class Foo(object):
...     __slots__ = ('spam',)
... 
>>> dir(Foo())
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', 'spam']
>>> Foo().spam
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: spam
>>> Foo.spam
<member 'spam' of 'Foo' objects>
>>> type(Foo.spam)
<type 'member_descriptor'>

So python still has to look at the class for each attribute access on an instance of Foo (to find the descriptor). Any unknown attribute (say, Foo.ham) will still result in Python looking through the class MRO to search for that attribute, and that includes dictionary searches. And you can still assign additional attributes to the class:

>>> Foo.ham = 'eggs'
>>> dir(Foo)
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__slots__', '__str__', '__subclasshook__', 'ham', 'spam']
>>> Foo().ham
'eggs'

The slot descriptors are created when the class is created, and access memory assigned to each instance to store and retrieve a reference to the associated value (the same chunk of memory that tracks instance reference counts and a reference back to the class object). Without slots, a descriptor for __dict__ is used accessing a reference to a dict object in the same manner.

Martijn Pieters
  • 1,048,767
  • 296
  • 4,058
  • 3,343
  • 9
    *"not significantly"*? 10% improvement for a single line change is quite significant in my book.If you understand `__slots__`'s obscure side-effects, of course :) – MestreLion Aug 06 '14 at 17:50
  • See also: [`http://code.activestate.com/recipes/532903-how-__slots__-are-implemented/`](http://code.activestate.com/recipes/532903-how-__slots__-are-implemented/) – Mr_and_Mrs_D Dec 13 '16 at 22:17
  • For the curious: if you don't use `__slots__` and instead declare the member at the class level, you'll see a speed _penalty_ of about 5% over the plain non-slots version. – z0r Apr 05 '17 at 00:48
  • With instantiation the difference becomes (unsurprisingly) wider with non-slots version being almost 50% slower: `>>> timeit.timeit('Foo().spam', 'from __main__ import Foo;') 0.33598813007120043 >>> timeit.timeit('Bar().spam', 'from __main__ import Bar;') 0.49041724496055394` – voneiden Jan 30 '22 at 11:52
  • Guido (the guy who created \_\_slots\_\_ and Python in general), explicitly states that he created \_\_slots\_\_ for the purpose of reducing access time to attributes by avoiding the class dictionary lookup needed to check for descriptors before any attribute is accessed. See the \_\_slots\_\_ section on his blog here: https://python-history.blogspot.com/2010/06/inside-story-on-new-style-classes.html – Matviy Kotoniy Oct 12 '22 at 23:35
  • @MatviyKotoniy: Guido initially came up with the idea for `__slots__` to counteract the expected performance hit introduced by the checks for descriptors, but that turned out to *not be an issue.* As stated in that same blog post: *This turned out unnecessary, but by that time it was of course too late to remove `__slots__`. Of course, used properly, slots really can increase performance, **especially by reducing memory footprint when many small objects are created.*** (bold emphasis mine). – Martijn Pieters Nov 25 '22 at 17:37
7

It might speed up a program where you instantiate lots of objects of the same class, genuinely never change what attributes they have, and cache misses on all those duplicate dictionaries present a real performance problem.

This is really just a special case of the general situation where saving space sometimes saves time as well, where cache is the limiting factor.

So, it probably won't make accessing one object faster, but may speed up accessing many objects of the same type.

See also this question.

Community
  • 1
  • 1
Useless
  • 64,155
  • 6
  • 88
  • 132