How to pass arguments to the metaclass from the class definition in Python 3.x?

Question

This is a Python 3.x version of the How to pass arguments to the metaclass from the class definition? question, listed separately by request since the answer is significantly different from Python 2.x.

---

In Python 3.x, how do I pass arguments to a metaclass's __prepare__, __new__, and __init__ functions so a class author can give input to the metaclass on how the class should be created?

As my use case, I'm using metaclasses to enable automatic registration of classes and their subclasses into PyYAML for loading/saving YAML files. This involves some extra runtime logic not available in PyYAML's stock YAMLObjectMetaClass. In addition, I want to allow class authors to optionally specify the tag/tag-format-template that PyYAML uses to represent the class and/or the function objects to use for construction and representation. I've already figured out that I can't use a subclass of PyYAML's YAMLObjectMetaClass to accomplish this--"because we don't have access to the actual class object in __new__" according to my code comment--so I'm writing my own metaclass that wraps PyYAML's registration functions.

Ultimately, I want to do something along the lines of:

from myutil import MyYAMLObjectMetaClass

class MyClass(metaclass=MyYAMLObjectMetaClass):
    __metaclassArgs__ = ()
    __metaclassKargs__ = {"tag": "!MyClass"}

...where __metaclassArgs__ and __metaclassKargs__ would be arguments going to the __prepare__, __new__, and __init__ methods of MyYAMLObjectMetaClass when the MyClass class object is getting created.

Of course, I could use the "reserved attribute names" approach listed in the Python 2.x version of this question, but I know there is a more elegant approach available.

Answer 1

After digging through Python's official documentation, I found that Python 3.x offers a native method of passing arguments to the metaclass, though not without its flaws.

Simply add additional keyword arguments to your class declaration:

class C(metaclass=MyMetaClass, myArg1=1, myArg2=2):
  pass

...and they get passed into your metaclass like so:

class MyMetaClass(type):

  @classmethod
  def __prepare__(metacls, name, bases, **kargs):
    #kargs = {"myArg1": 1, "myArg2": 2}
    return super().__prepare__(name, bases, **kargs)

  def __new__(metacls, name, bases, namespace, **kargs):
    #kargs = {"myArg1": 1, "myArg2": 2}
    return super().__new__(metacls, name, bases, namespace)
    #DO NOT send "**kargs" to "type.__new__".  It won't catch them and
    #you'll get a "TypeError: type() takes 1 or 3 arguments" exception.

  def __init__(cls, name, bases, namespace, myArg1=7, **kargs):
    #myArg1 = 1  #Included as an example of capturing metaclass args as positional args.
    #kargs = {"myArg2": 2}
    super().__init__(name, bases, namespace)
    #DO NOT send "**kargs" to "type.__init__" in Python 3.5 and older.  You'll get a
    #"TypeError: type.__init__() takes no keyword arguments" exception.

You have to leave kargs out of the call to type.__new__ and type.__init__ (Python 3.5 and older; see "UPDATE" below) or will get you a TypeError exception due to passing too many arguments. This means that--when passing in metaclass arguments in this manner--we always have to implement MyMetaClass.__new__ and MyMetaClass.__init__ to keep our custom keyword arguments from reaching the base class type.__new__ and type.__init__ methods. type.__prepare__ seems to handle the extra keyword arguments gracefully (hence why I pass them through in the example, just in case there's some functionality I don't know about that relies on **kargs), so defining type.__prepare__ is optional.

UPDATE

In Python 3.6, it appears type was adjusted and type.__init__ can now handle extra keyword arguments gracefully. You'll still need to define type.__new__ (throws TypeError: __init_subclass__() takes no keyword arguments exception).

Breakdown

In Python 3, you specify a metaclass via keyword argument rather than class attribute:

class MyClass(metaclass=MyMetaClass):
  pass

This statement roughly translates to:

MyClass = metaclass(name, bases, **kargs)

...where metaclass is the value for the "metaclass" argument you passed in, name is the string name of your class ('MyClass'), bases is any base classes you passed in (a zero-length tuple () in this case), and kargs is any uncaptured keyword arguments (an empty dict {} in this case).

Breaking this down further, the statement roughly translates to:

namespace = metaclass.__prepare__(name, bases, **kargs)  #`metaclass` passed implicitly since it's a class method.
MyClass = metaclass.__new__(metaclass, name, bases, namespace, **kargs)
metaclass.__init__(MyClass, name, bases, namespace, **kargs)

...where kargs is always the dict of uncaptured keyword arguments we passed in to the class definition.

Breaking down the example I gave above:

class C(metaclass=MyMetaClass, myArg1=1, myArg2=2):
  pass

...roughly translates to:

namespace = MyMetaClass.__prepare__('C', (), myArg1=1, myArg2=2)
#namespace={'__module__': '__main__', '__qualname__': 'C'}
C = MyMetaClass.__new__(MyMetaClass, 'C', (), namespace, myArg1=1, myArg2=2)
MyMetaClass.__init__(C, 'C', (), namespace, myArg1=1, myArg2=2)

Most of this information came from Python's Documentation on "Customizing Class Creation".

Answer 2

Here's a version of the code from my answer to that other question about metaclass arguments which has been updated so that it'll work in both Python 2 and 3. It essentially does the same thing that Benjamin Peterson's six module's with_metaclass() function does — which namely is to explicitly create a new base class using the desired metaclass on-the-fly, whenever needed and thereby avoiding errors due to the metaclass syntax differences between the two versions of Python (because the way to do that didn't change).

from __future__ import print_function
from pprint import pprint

class MyMetaClass(type):
    def __new__(cls, class_name, parents, attrs):
        if 'meta_args' in attrs:
            meta_args = attrs['meta_args']
            attrs['args'] = meta_args[0]
            attrs['to'] = meta_args[1]
            attrs['eggs'] = meta_args[2]
            del attrs['meta_args'] # clean up
        return type.__new__(cls, class_name, parents, attrs)

# Creates base class on-the-fly using syntax which is valid in both
# Python 2 and 3.
class MyClass(MyMetaClass("NewBaseClass", (object,), {})):
    meta_args = ['spam', 'and', 'eggs']

myobject = MyClass()

pprint(vars(MyClass))
print(myobject.args, myobject.to, myobject.eggs)

Output:

dict_proxy({'to': 'and', '__module__': '__main__', 'args': 'spam',
            'eggs': 'eggs', '__doc__': None})
spam and eggs

Answer 3

In Python 3, you specify a metaclass via keyword argument rather than class attribute:

It's worth to say, that this style is not backward compatible to python 2. If you want to support both python 2 and 3, you should use:

from six import with_metaclass
# or
from future.utils import with_metaclass

class Form(with_metaclass(MyMetaClass, object)):
    pass

Answer 4

Here's the simplest way to pass arguments to a metaclass in Python 3:

Python 3.x

class MyMetaclass(type):
    def __new__(mcs, name, bases, namespace, **kwargs):
        return super().__new__(mcs, name, bases, namespace)

    def __init__(cls, name, bases, namespace, custom_arg='default'):
        super().__init__(name, bases, namespace)

        print('Argument is:', custom_arg)


class ExampleClass(metaclass=MyMetaclass, custom_arg='something'):
    pass

You can also create a base class for metaclasses that only use __init__ with extra arguments:

class ArgMetaclass(type):
    def __new__(mcs, name, bases, namespace, **kwargs):
        return super().__new__(mcs, name, bases, namespace)


class MyMetaclass(ArgMetaclass):
    def __init__(cls, name, bases, namespace, custom_arg='default'):
        super().__init__(name, bases, namespace)

        print('Argument:', custom_arg)


class ExampleClass(metaclass=MyMetaclass, custom_arg='something'):
    pass