Using metaclass to keep track of instances in python

Question

I need to keep tracks of instances of some classes (and do other stuff with those classes). I would like to not have to declare any extra code in the classes in question, thus everything should ideally be handled in the metaclass.

What I can't figure out is how to add a weak reference to each new instance of those classes. For example:

class Parallelizable(type):
    def __new__(cls, name, bases, attr):
        meta = super().__new__(cls, name, bases, attr)
        # storing the instances in this WeakSet
        meta._instances = weakref.WeakSet()
        return meta

    @property
    def instances(cls):
        return [x for x in cls._instances]

class Foo(metaclass=Parallelizable)
    def __init__(self, name):
        super().__init__()
        self.name = name

        # I would like to avoid having to do that - instead have the metaclass manage it somehow
        self._instances.add(self)

Any ideas? I can't seem to find a hook on the metaclass side to get into the __init__ of Foo....

Slightly more generalised similar question at https://stackoverflow.com/questions/12101958/how-to-keep-track-of-class-instances/63758584 — John Vandenberg, Jun 26 '22 at 05:40

jsbueno · Accepted Answer · 2019-07-07T12:23:28.027

The method on the metaclass that is called when each new instance of its "afiliated" classes is __call__. If you put the code to record the instances in there, that is all the work you need:


from weakref import WeakSet

# A convenient class-level descriptor to retrieve the instances:

class Instances:
    def __get__(self, instance, cls):
        return [x for x in cls._instances]

class Parallelizable(type):
    def __init__(cls, name, bases, attrs, **kw):
        super().__init__(name, bases, attrs, **kw)
        cls._instances = WeakSet()
        cls.instances = Instances()

    def __call__(cls, *args, **kw):
        instance = super().__call__(*args, **kw)
        cls._instances.add(instance)
        return instance

The same code will work without the descriptor at all - it is just a nice way to have a class attribute that would report the instances. But if the WeakSet is enough, this code suffices:


from weakref import WeakSet
class Parallelizable(type):
    def __init__(cls, name, bases, attrs, **kw):
        super().__init__(name, bases, attrs, **kw)
        cls.instances = WeakSet()

    def __call__(cls, *args, **kw):
        instance = super().__call__(*args, **kw)
        cls.instances.add(instance)
        return instance

Why use the class-level descript as opposed to a decorator in the metaclass? Any pros/cons? — logicOnAbstractions, Jul 07 '19 at 02:43
The descriptor is just a nice to have, and not related to the record--keeping at all. What is different from the decorator here is that the tracking code is in the `__call__` method, and it is a whole order of magnitude simpler than the decorator approach - and will work when the target class does not have an `__init__` method, for example, without any modifications. — jsbueno, Jul 07 '19 at 12:21
I recommend creating a class for easy subclassing (rename `Parallelizable` above to `ParallelizableMeta`) then: `class Parallelizable(metaclass=ParallelizableMeta): pass ` — run_the_race, Oct 20 '22 at 13:39

unutbu · Answer 2 · 2019-07-07T02:50:43.337

2

You could decorate the attrs['__init__'] method in Parallizable.__new__:

import weakref
import functools
class Parallelizable(type):
    def __new__(meta, name, bases, attrs):
        attrs['__init__'] = Parallelizable.register(attrs['__init__'])
        cls = super().__new__(meta, name, bases, attrs)
        cls._instances = weakref.WeakSet()
        return cls

    @classmethod
    def register(cls, method):
        @functools.wraps(method)
        def newmethod(self, *args, **kwargs):
            method(self, *args, **kwargs)
            self._instances.add(self)
        return newmethod

    @property
    def instances(cls):
        return [x for x in cls._instances]

class Foo(metaclass=Parallelizable):
    def __init__(self, name):
        "Foo.__init__ doc string"
        super().__init__()
        self.name = name

# Notice that Foo.__init__'s docstring is preserved even though the method has been decorated
help(Foo.__init__)
# Help on function __init__ in module __main__:
#
# __init__(self, name)
#     Foo.__init__ doc string

stilton = Foo('Stilton')
gruyere = Foo('Gruyere')
print([inst.name for inst in Foo.instances])
# ['Gruyere', 'Stilton']

del stilton
print([inst.name for inst in Foo.instances])
# ['Gruyere']

edited Jul 07 '19 at 02:50

answered Jul 07 '19 at 02:08

unutbu

842,883
184
1,785
1,677

Auto-applying decorators is a nice approach, but then, there are corner-cases such as, when the class has no `__init__` of its own. – jsbueno Jul 07 '19 at 02:35
there is a bug there - `cls` in your `__new__` refers to the metaclass itself (and "meta" the new class created with the metaclass) - so, each new class created with the code as is will reset the register. – jsbueno Jul 07 '19 at 02:38
2

@jsbueno: Yes, thanks for the correction regarding the `meta`/`cls` naming mixup. Regarding `__init__`: There is more than one way to handle classes with no `__init__` (such as raise an exception, add a trivial registering `__init__`, or do nothing.) Since the OP did not say how he wished this to be handled, I chose one which would raise an exception -- alerting anyone using Parallelizable with an `__init__`-less class that the case needs to be handled. – unutbu Jul 07 '19 at 02:54
That's an interesting approach. I would never have come up with that. I do understand what you're doing after playing with it some. However, why would you see as the main benefit of using this approach vs _ _call_ _? Other than enriching my understanding of the finer workings of Python? – logicOnAbstractions Jul 07 '19 at 02:57
1

@Francky_V: I like @jsbueno's solution better than my own because it is very readable, easy to understand, and handles `__init__`-less classes without fuss. – unutbu Jul 07 '19 at 03:12
Thanks for being upfront. I still learned something from your answer that being said! – logicOnAbstractions Jul 07 '19 at 23:44

score 0 · Answer 3 · answered Oct 20 '22 at 13:59

How about this, its a class to inherit from, instead of a metaclass. I think its simpler but achieves the same point:

class AutoDiscovered:
    instances = []

    def __new__(cls, *args, **kwargs):
        obj = super().__new__(cls)
        cls.instances.append(obj)
        return obj

Usage:

class Foo(AutoDiscovered):
    pass

a = Foo()
b = Foo()
print(Foo.instances)  # [<__main__.Foo object at 0x7fdabd345430>, <__main__.Foo object at 0x7fdabd345370>]

Using metaclass to keep track of instances in python

3 Answers3