Decorating a method that's already a classmethod?

Question

I had an interesting problem this morning. I had a base class that looked like this:

# base.py
class Base(object):

    @classmethod
    def exists(cls, **kwargs):
        # do some work
        pass

And a decorator module that looked like this:

# caching.py

# actual caching decorator
def cached(ttl):
    # complicated

def cached_model(ttl=300):
    def closure(model_class):
        # ...
        # eventually:
        exists_decorator = cached(ttl=ttl)
        model_class.exists = exists_decorator(model_class.exists))

        return model_class
    return closure

Here's my subclass model:

@cached_model(ttl=300)
class Model(Base):
    pass

Thing is, when I actually call Model.exists, I get complaints about the wrong number of arguments! Inspecting the arguments in the decorator shows nothing weird going on - the arguments are exactly what I expect, and they match up with the method signature. How can I add further decorators to a method that's already decorated with classmethod?

Not all models are cached, but the exists() method is present on every model as a classmethod, so re-ordering the decorators isn't an option: cached_model can add classmethod to exists(), but then what makes exists() a classmethod on uncached models?

Answer 1

In Python, when a method is declared, in a function body, it is exactly like a function - once the class is parsed and exists, retrieving the method through the "." operator transforms that function - on the fly - into a method. This transform does add the first parameter to the method (if it is not an staticmethod) -

so:

>>> class A(object):
...    def b(self):
...        pass
... 
>>> A.b is A.b
False

Becasue each retrieving of the "b" attribute of "A" yields a different instance of the "method object b"

>>> A.b
<unbound method A.b>

The original function "b" can be retrieved without any trasnform if one does

>>> A.__dict__["b"]
<function b at 0xe36230>

For a function decorated with @classmethod just the same happens, and the value "class" is added to the parameter list when it is retrieved from A.

The @classmethod and @staticmethod decorators will wrap the underlying function in a different descriptor than the normal instancemethod. A classmethod object - which is what a function becomes when it is wrapped with classmethod is a descriptor object, which has a '__get__' method which will return a function wrapping the underlying function - and adding the "cls" parameter before all the other ones.

Any further decorator to a @classmethod has to "know" it is actually dealing with a descriptor object, not a function. -

>>> class A(object):
...    @classmethod
...    def b(cls):
...       print b
... 
>>> A.__dict__["b"]
<classmethod object at 0xd97a28>

So, it is a lot easier to let the @classmethod decorator to be the last one to be applied to the method (the first one on the stack) - so that the other decorators work on a simple function (knowing that the "cls" argument will be inserted as the first one).

Answer 2

Thanks to jsbueno for the information about Python. I was looking for an answer to this question based on the case of decorating all methods of a class. Based on looking for an answer to this question and jsbueno's reponse, I was able to gather something along the lines of:

def for_all_methods(decorator):

    def decorate(cls):

        for attr in dir(cls):
            possible_method = getattr(cls, attr)
            if not callable(possible_method):
                continue

            # staticmethod
            if not hasattr(possible_method, "__self__"):
                raw_function = cls.__dict__[attr].__func__
                decorated_method = decorator(raw_function)
                decorated_method = staticmethod(decorated_method)

            # classmethod
            elif type(possible_method.__self__) == type:
                raw_function = cls.__dict__[attr].__func__
                decorated_method = decorator(raw_function)
                decorated_method = classmethod(decorated_method)

            # instance method
            elif possible_method.__self__ is None:
                decorated_method = decorator(possible_method)

            setattr(cls, attr, decorated_method)

        return cls
    return decorate

There's a bit of redundancy and a few variations you could use to chop this down a bit.

Answer 3

The classmethod decorator actually prepends a class argument to calls to the method, in certain circumstances, as far as I can tell, in addition to binding the method to the class. The solution was editing my class decoration closure:

def cached_model(ttl=300):
    def closure(model_class):
        # ...
        # eventually:
        exists_decorator = cached(ttl=ttl, cache_key=exists_cache_key)
        model_class.exists = classmethod(exists_decorator(model_class.exists.im_func))

        return model_class
    return closure

The im_func property appears to get a reference to the original function, which allows me to reach in and decorate the original function with my caching decorator, and then wrap that whole mess in a classmethod call. Summary, classmethod decorations are not stackable, because arguments seem to be injected.

Answer 4

Just a functional example to add to Scott Lobdell's great answer...

messages.py

from distutils.cmd import Command

import functools
import unittest

def for_all_methods(decorator):

    def decorate(cls):

        for attr in cls.__dict__:
            possible_method = getattr(cls, attr)
            if not callable(possible_method):
                continue

            # staticmethod
            if not hasattr(possible_method, "__self__"):
                raw_function = cls.__dict__[attr].__func__
                decorated_method = decorator(raw_function)
                decorated_method = staticmethod(decorated_method)

            # classmethod
            if type(possible_method.__self__) == type:
                raw_function = cls.__dict__[attr].__func__
                decorated_method = decorator(raw_function)
                decorated_method = classmethod(decorated_method)


            # instance method
            elif possible_method.__self__ is None:
                decorated_method = decorator(possible_method)

            setattr(cls, attr, decorated_method)

        return cls

    return decorate

def add_arguments(func):
    """
    The add_arguments decorator simply add the passed in arguments
    (args and kwargs) the returned error message.
    """    
    @functools.wraps(func)
    def wrapped(self, *args, **kwargs):
        try:
            message = func(self, *args, **kwargs)
            message = ''.join([message, 
                               "[ args:'", str(args), "'] ", 
                               "[ kwargs:'", str(kwargs), "' ] " 
                               ])
            return message

        except Exception as e:
            err_message = ''.join(["errorhandler.messages.MESSAGE: '",
                                   str(func), 
                                   "(", str(args), str(kwargs), ")' ", 
                                   "FAILED FOR UNKNOWN REASON. ",
                                   " [ ORIGINAL ERROR: ", str(e), " ] "
                                   ])
            return err_message

    return wrapped



@for_all_methods(add_arguments)    
class MESSAGE(object):
    """
            log.error(MSG.triggerPhrase(args, kwargs))

    """    
    @classmethod
    def TEMPLATE(self, *args, **kwargs):
        message = "This is a template of a pre-digested message."
        return message

usage

from messages import MESSAGE

if __name__ == '__main__':
    result = MESSAGE.TEMPLATE(1,2,test=3)
    print result

output

This is a template of a pre-digested message.[ args:'(1, 2)'] [ kwargs:'{'test': 3}' ]