2

I read a bit on python's object attribute lookup (here: https://blog.ionelmc.ro/2015/02/09/understanding-python-metaclasses/#object-attribute-lookup).

Seems pretty straight forward, so I tried it out (python3):

class A:
    def __getattr__(self, attr):
        return (1,2,3)

a = A()

a.foobar #returns (1,2,3) as expected
a.__getattribute__('foobar') # raises AttributeError

My question is, aren't the two supposed to be identical?

Why does the second one raise an attribute error?

So apparently the answer is that the logic for a.foobar IS different from the logic for a.__getattribute("foobar"). According to the data model: a.foobar calls a.__getattribute("foobar") and if it raises an AttributeError, it calls a.-__getattr__('foobar')

So it seems the article has a mistake in their diagram. Is this correct?

And another question: Where does the real logic for a.foobar sit? I thought it was in __getattribute__ but apparently not entirely.

Edit: Not a duplicate of

Difference between __getattr__ vs __getattribute__. I am asking here what is the different between object.foo and object.__getattribute__("foo"). This is different from __getattr__ vs __getatribute__ which is trivial...

Tonechas
  • 13,398
  • 16
  • 46
  • 80
deller
  • 490
  • 3
  • 12
  • 1
    Refer to official doc first: https://docs.python.org/3.5/reference/datamodel.html#object.__getattribute__ – thodnev Aug 19 '16 at 16:38
  • Change `a.__getattribute__('foobar')` to `a.__getattr__('foobar')` and see what happens. – DeepSpace Aug 19 '16 at 16:38
  • Not interested in that... `__getattribute__` is supposed to eventually call `__getattr__`. `a.foobar` is supposed to call `__getattribute__` – deller Aug 19 '16 at 16:40
  • Why? It doesn't exist so `__getattr__` is invoked instead. – vaultah Aug 19 '16 at 16:41
  • @thodnev the data model spec is unclear. Does this mean that `a.foo` is implemented like this: `try a.__getattribute__ except AtributeError: a.__getattr__` ? – deller Aug 19 '16 at 16:44
  • @dellar More or less. `__getattr__` is only tried if it exists, and there are special exceptions when magic methods are used. For instance, `len` bypasses both `__getattribute__` and `__getattr__` to retrieve `__len__`. – Dunes Aug 19 '16 at 16:50
  • @deller the `__getattribute__` is a decriptor, defined in one or more base classes (or in `object` if not elsewhere). By default it invokes `__getattr__` for unknown (not present in `__dict__` or nondefined as `__slots__` attrs). So usually you need to redefine `__getattr__` to intercept unknown attributes, and `__getattribute__` gives you total control (regulary not needed). Something like it – thodnev Aug 19 '16 at 16:51
  • 2
    @thodnev: `__getattribute__` doesn't invoke `__getattr__`; the fallback to `__getattr__` is done separately. The fact that this is not `__getattribute__`'s responsibility is probably the source of the questioner's confusion. – user2357112 Aug 19 '16 at 16:54
  • @user2357112 _If the class also defines `__getattr__()`, the latter will not be called unless `__getattribute__()` either calls it explicitly or raises an AttributeError_ – thodnev Aug 19 '16 at 16:56
  • @user2357112 This is consistent with what i tried. Does this mean the diagram on the article is mistaken? it shows `__getattribute_`_calling `__getattr__` – deller Aug 19 '16 at 16:57
  • @thodnev: That last bit about "or raises an AttributeError" is key. If `__getattribute__` raises an AttributeError, the attribute lookup process will try `__getattr__` next. – user2357112 Aug 19 '16 at 16:58
  • Where is all of this logic implemented? i thought `__getattribute__` was in charge of ALL the lookup process... – deller Aug 19 '16 at 16:59
  • @user2357112 Do you guys believe this is a really a duplicate? Why was it marked as such? how can i remove the duplicate? i think its a legitimate question... – deller Aug 19 '16 at 17:01
  • @vaultah please take a look. I do not think this is a duplicate... – deller Aug 19 '16 at 17:02

1 Answers1

9

It's easy to get the impression that __getattribute__ is responsible for more than it really is. thing.attr doesn't directly translate to thing.__getattribute__('attr'), and __getattribute__ is not responsible for calling __getattr__.

The fallback to __getattr__ happens in the part of the attribute access machinery that lies outside __getattribute__. The attribute lookup process works like this:

  • Find the __getattribute__ method through a direct search of the object's type's MRO, bypassing the regular attribute lookup process.
  • Try __getattribute__.
    • If __getattribute__ returned something, the attribute lookup process is complete, and that's the attribute value.
    • If __getattribute__ raised a non-AttributeError, the attribute lookup process is complete, and the exception propagates out of the lookup.
    • Otherwise, __getattribute__ raised an AttributeError. The lookup continues.
  • Find the __getattr__ method the same way we found __getattribute__.
    • If there is no __getattr__, the attribute lookup process is complete, and the AttributeError from __getattribute__ propagates.
  • Try __getattr__, and return or raise whatever __getattr__ returns or raises.

At least, in terms of the language semantics, it works like that. In terms of the low-level implementation, some of these steps may be optimized out in cases where they're unnecessary, and there are C hooks like tp_getattro that I haven't described. You don't need to worry about that kind of thing unless you want to dive into the CPython interpreter source code.

user2357112
  • 260,549
  • 28
  • 431
  • 505
  • TL;DR `__getattribute__` bypasses the regular attribute lookup only if it is defined and is not intended to call `__getattr__`. Calling `__getattribute__` directly on a class which does not define it, falls back to `object.__getattribute__` or another base class and thus raises (probably) an `AttributeError` – code_onkel Aug 19 '16 at 18:16