More arguments in derived class __init__ than base class __init__

Question

How can I extend the __init__ of a base class, add more arguments to be parsed, without requiring super().__init__(foo, bar) in every derived class?

class Ipsum:
    """ A base ipsum instance. """

    def __init__(self, foo, bar):
        self.foo = flonk(foo)
        grungole(self, bar)
        self._baz = make_the_baz()


class LoremIpsum(Ipsum):
    """ A more refined ipsum that also lorems. """

    def __init__(self, foo, bar, dolor, sit, amet):
        super().__init__(foo, bar)
        farnark(sit, self, amet)
        self.wibble(dolor)

The purpose of the example is to show that there is significant processing happening in the Ipsum.__init__, so it should not be duplicated in every sub-class; and the LoremIpsum.__init__ needs the foo and bar parameters processed exactly the same as Ipsum, but also has its own special parameters.

It also shows that, if Ipsum needs to be modified to accept a different signature, every derived class also needs to change not only its signature, but how it calls the superclass __init__. That's unacceptably fragile.

Instead, I'd like to do something like:

class Ipsum:
    """ A base ipsum instance. """

    def __init__(self, foo, bar, **kwargs):
        self.foo = flonk(foo)
        grungole(self, bar)
        self._baz = make_the_baz()

        self.parse_init_kwargs(kwargs)

    def parse_init_kwargs(self, kwargs):
        """ Parse the remaining kwargs to `__init__`. """
        pass


class LoremIpsum(Ipsum):
    """ A more refined ipsum that also lorems. """

    def parse_init_kwargs(self, kwargs):
        (dolor, sit, amet) = (kwargs['dolor'], kwargs['sit'], kwargs['amet'])
        farnark(sit, self, amet)
        self.wibble(dolor)

That has the big advantage that LoremIpsum need only do the parts that are special to that class; handling Ipsum arguments is handled by that class's __init__ without any extra code.

The disadvantage is obvious, though: this is effectively re-implementing the handling of named parameters by passing a dictionary around. It avoids a lot of repetition, but isn't very clear.

What tools are available to avoid the sub-class definitions always needing to declare the foo and bar parameters, and always needing to call super().__init__(foo, bar)? Those are easy to get wrong, so it would be better if they weren't needed and could just automatically happen, while still allowing LoremIpsum's customisation of the initialisation.

Answer 1

A flexible approach is to have every method in the ancestor tree cooperatively designed to accept keyword arguments and a keyword-arguments dictionary, to remove any arguments that it needs, and to forward the remaining arguments using **kwds, eventually leaving the dictionary empty for the final call in the chain.

Each level strips-off the keyword arguments that it needs so that the final empty dict can be sent to a method that expects no arguments at all (for example, object.__init__ expects zero arguments):

class Shape:
    def __init__(self, shapename, **kwds):
        self.shapename = shapename
        super().__init__(**kwds)        

class ColoredShape(Shape):
    def __init__(self, color, **kwds):
        self.color = color
        super().__init__(**kwds)

cs = ColoredShape(color='red', shapename='circle')

For more on this approach, see the "Practical Advice" section of the Super Considered Super blogpost, or see the related at Pycon video.

In your example, the code would look like this:

class Ipsum:
    """ A base ipsum instance. """

    def __init__(self, foo, bar):
        self.foo = flonk(foo)
        grungole(self, bar)
        self._baz = make_the_baz()

class LoremIpsum(Ipsum):
    """ A more refined ipsum that also lorems. """

    def __init__(self, dolor, sit, amet, **kwds):
        super().__init__(**kwds)
        farnark(sit, self, amet)
        self.wibble(dolor)

An instantiation would look like this:

li = LoremIpsum(dolor=1, sit=2, amet=3, foo=4, bar=5) 

Note, this lets you achieve your original objective of adding new arguments to either __init__ method without affecting the other.

Answer 2

The usual way to write these is roughly as follows:

class Ipsum:  # 3.x-ism; in 2.x always inherit from object.
    def __init__(self, arg1, arg2, arg3):
        # etc...

class LoremIpsum(Ipsum):
    def __init__(self, arg4, arg5, *args, **kwargs):
        super().__init__(*args, **kwargs)  # 3.x-ism; in 2.x super() needs arguments.
        # Do stuff with arg4 and arg5

This does not require modifying derived classes when the base class changes signatures. You will still need to modify everything that directly instantiates your base class, so changing the signature is still not something you want to do frequently.

This approach is also superior because it will generally behave more or less correctly in the case of multiple inheritance, even if your derived class gets put in front of another derived class in the method resolution order and you've never heard of that other class (in other words, even if super().__init__() is calling a method you know nothing about). This is discussed in detail in Hettinger's super() considered super blog post.

Answer 3

Derived classes need to call the superclass __init__. That's how object-oriented programming works. Is it brittle? Yes, it can be; that's why deep or broad class hierarchies aren't usually a good idea. If the operations on foo and bar in Lorem are significant (as you say they are), then correctly invoking Lorem.__init__() is a cost that must be paid. Default values for foo and bar (and any new parameters) are the only tool I can think of that might make change the amount of textual change needed, but default values aren't always appropriate, so that mitigation might not apply.