How to Iterate over superclasses in multilevel inheritance?

Question

I'm designing a metaclass to overwrite class __call__ function, preceding it with the execution of its superclasses __call__ recursively. The idea is be able to get the following result for the code below:

Abstract
Base
Super
Child

class Abstract(metaclass=Meta):
    def __call__(self):
        print("Abstract")

class Base(Abstract):
    def __call__(self):
        print("Base")

class Super(Abstract):
    def __call__(self):
        print("Super")

class Parent:
    def __call__(self):
        print("Parent")

class Child(Base, Super, Parent):
    def __call__(self):
        print("Child")

So far I was writing my Meta.new as the following:

def __new__(meta, name, bases, attr):

    __call__ = attr['__call__']

    def recursive_call(self):
        for cls in [base for base in self.__class__.__bases__ if type(base) is Meta]:
            cls.__call__(super(cls, self))
        __call__(self)

    attr['__call__'] = recursive_call

    return super(Meta, meta).__new__(
        meta, name, bases, attr
    )

And it's actually working for one-level-inheritance classes, however it isn't for multilevel ones.

How could I fix this code to achieve my goal? Or would there be an easier way to hit it, rather then metaclasses?

That seems like a very questionable way to design a metaclass. — user2357112, May 03 '20 at 02:26
If you really want `__call__` to behave like this, it'd be better to have a separate `_call` hook and have a base class with a `__call__` that calls all the `_call` methods, but having `__call__` work like this seems weird on its own. — user2357112, May 03 '20 at 02:30
What is this: `cls.__call__(super(cls, self))` supposed to do?? — juanpa.arrivillaga, May 03 '20 at 02:30
`cls.__call__(super(cls, self))` was the way I tried to pass the *superclass object* to its `__call__`, since if I passed `self` the *baseclasses list* come repeated and I got an `RecursionError: maximum recursion depth exceeded while calling a Python object ` — artu-hnrq, May 03 '20 at 02:34
`super(cls, self)` will not return the super-class object. It will return a `super` object. — juanpa.arrivillaga, May 03 '20 at 02:35
Yeah, now I got it! But couldn't figure out how to work with the proper object yet — artu-hnrq, May 03 '20 at 02:39

score 1 · Accepted Answer · edited May 04 '20 at 17:00

What is preventing your desired results is that you are iterating over the class' __bases__ - these only list the immediate superclasses. If you change your metacass to iterate over __mro__, Python's linearized sequence of all of one class' ancestors, it will work:


In [14]: class Abstract(metaclass=Meta): 
    ...:     def __call__(self): 
    ...:         print("Abstract") 
    ...:  
    ...: class Base(Abstract): 
    ...:     def __call__(self): 
    ...:         print("Base") 
    ...:  
    ...: class Super(Abstract): 
    ...:     def __call__(self): 
    ...:         print("Super") 
    ...:  
    ...: class Parent: 
    ...:     def __call__(self): 
    ...:         print("Parent") 
    ...:  
    ...: class Child(Parent): 
    ...:     def __call__(self): 
    ...:         print("Child") 
    ...:                                                                                                            

In [15]: Child.__mro__                                                                                              
Out[15]: (__main__.Child, __main__.Parent, object)

Anyway, this turns out to be a bit trickier than seem at first glance - there are corner cases - what if one of your eligible classes do not feature a __call__ for example? What if one of the methods do include an ordinary "super()" call? Ok, add a marker to avoid unwanted re-entrancy in cases one does put a "super()" - what if it is running in a multithreaded environment and two instances are being creaed at the same time?

All in all, one has to make the correct combination of using Python's attribute fetching mechanisms - to pick the methods in the correct instances. I made a choice of copying the original __call__ method to anothr method in the class itself, so it can not only store the original method, but also work as a marker for the eligible classes.

Also, note that this works just the same for __call__ as it would work for any other method - so I factored the name "__call__" to a constant to ensure that (and it could be made a list of methods, or all methods whose name have a certain prefix, and so on).


from functools import wraps
from threading import local as threading_local  

MARKER_METHOD = "_auto_super_original"
AUTO_SUPER = "__call__"

class Meta(type):
    def __new__(meta, name, bases, attr):

        original_call = attr.pop(AUTO_SUPER, None)

        avoid_rentrancy = threading_local()
        avoid_rentrancy.running = False

        @wraps(original_call)
        def recursive_call(self, *args, _wrap_call_mro=None, **kwargs):
            if getattr(avoid_rentrancy, "running", False):
                return
            avoid_rentrancy.running = True

            mro = _wrap_call_mro or self.__class__.__mro__

            try:
                for index, supercls in enumerate(mro[1:], 1):
                    if MARKER_METHOD in supercls.__dict__:
                        supercls.__call__(self, *args, _wrap_call_mro=mro[index:], **kwargs)
                        break
                getattr(mro[0], MARKER_METHOD)(self, *args, **kwargs)    
            finally:
                avoid_rentrancy.running = False

        if original_call:
            attr[MARKER_METHOD] = original_call
            attr[AUTO_SUPER] = recursive_call

        return super().__new__(
            meta, name, bases, attr
        )

ANd this is working on the console - I added a few more intermediate classes to cover for the corner-cases:


class Abstract(metaclass=Meta):
    def __call__(self):
        print("Abstract")

class Base1(Abstract):
    def __call__(self):
        print("Base1")

class Base2(Abstract):
    def __call__(self):
        print("Base2")

class Super(Base1):
    def __call__(self):
        print("Super")

class NonColaborativeParent():
    def __call__(self):
        print("Parent")

class ForgotAndCalledSuper(Super):
    def __call__(self):
        super().__call__()
        print("Forgot and called super")

class NoCallParent(Super):
    pass

class Child(NoCallParent, ForgotAndCalledSuper, Parent, Base2):
    def __call__(self):
        print("Child")

Result:

In [96]: Child()()                                                                                                  
Abstract
Base2
Base1
Super
Child
Forgot and called super
Child

Pretty englighter and aware of numerous unconsidered situations! — artu-hnrq, May 04 '20 at 15:19

score 0 · Answer 2 · answered May 03 '20 at 05:38

Well, I could find a solution taking advantage of class method resolution order (i.e its __mro__ attribute) and also following supports Monica's suggestion (thanks!).

My metaclass was like this:

class MetaComposition(type):
    def __new__(meta, name, bases, attr, __func__='__call__'):

        def __call__(self, *args, **kwargs):
            for cls in self.__class__.__compound__:
                cls.__run__(self, *args, **kwargs)

        attr['__run__'] = attr[__func__]
        attr[__func__] = __call__

        return super(MetaComposition, meta).__new__(meta, name, bases, attr)

    @property
    def __compound__(cls):
        return [
            element
            for element in
            cls.mro()[::-1]
            if type(element)
            is type(cls)
        ]

And this way the desired behavior is achieved

How to Iterate over superclasses in multilevel inheritance?

2 Answers2