How to register typing.Callable with Python @singledispatch?

Question

Background

Suppose I am to implement a simple decorator @notifyme that prints a message when the decorated function is invoked. I would like the decorator to accept one argument to print a customized message; the argument (along with the parentheses surrounding the argument) may be omitted, in which case the default message is printed:

@notifyme('Foo is invoked!')
def foo():
    pass

@notifyme  # instead of @notifyme()
def bar():
    pass

To allow the parentheses to be omitted, I have to provide two implementations of @notifyme:

The first implementation allows the user to customize the message, so it accepts a string as argument and returns a decorator:

def notifyme_customized(message: str) -> Callable[[Callable], Callable]:
    def decorator(func: Callable) -> Callable:
        def decorated_func(*args, **kwargs):
            print(str)
            return func(*args, **kwargs)
        return decorated_func
    return decorator

The second implementation is a decorator itself and uses the first implementation to print a default message:

def notifyme_default(func: Callable) -> Callable:
    return notifyme_customized('The function is invoked.')(func)

To make the two implementations above use the same name notifyme, I used functools.singledispatch to dynamically dispatch the call to notifyme to one of the two implementations:

# This is a complete minimal reproducible example

from functools import singledispatch
from typing import Callable

@singledispatch
def notifyme(arg):
    return NotImplemented

@notifyme.register
def notifyme_customized(message: str) -> Callable[[Callable], Callable]:
    def decorator(func: Callable) -> Callable:
        def decorated_func(*args, **kwargs):
            print(str)
            return func(*args, **kwargs)
        return decorated_func
    return decorator

@notifyme.register
def notifyme_default(func: Callable) -> Callable:
    return notifyme_customized('The function is invoked.')(func)

Problem

However, as the code is interpreted by the Python interpreter, it complains that typing.Callable is an invalid type:

Traceback (most recent call last):
  File "demo.py", line 20, in <module>
    def notifyme_default(func: Callable) -> Callable:
  File "C:\Program Files\Python38\lib\functools.py", line 860, in register
    raise TypeError(
TypeError: Invalid annotation for 'func'. typing.Callable is not a class.

I have found this issue on Python bug tracker, according to which it seems to be expected behavior since Python 3.7. Does a solution or workaround exist in Python 3.8 I use currently (or Python 3.9 that has been released recently)?

Thanks in advance.

Why don't you utilize a default argument instead of having the two functions? You're also using a generic without type parameters which may be why you're getting errors. You may want to use a `: function) -> function` annotation instead. — Maximilian Burszley, Oct 07 '20 at 17:13
@MaximilianBurszley Thank you for your comment! But if I use a default argument, I have to write `@notifyme()` when I want to use the default message, and I would like the parentheses be omitted as well (like `@notifyme`) — Yang Hanlin, Oct 07 '20 at 17:17
@MaximilianBurszley And if I use a `: function) -> function` the Python intepreter complains `NameError: name 'function' is not defined`, but `function` seems to be a built-in symbol so I am confused as well — Yang Hanlin, Oct 07 '20 at 17:19
Ah, I guess I should've tried the sample before giving feedback! I'll be back in a few with more comments since I'm curious now. — Maximilian Burszley, Oct 07 '20 at 17:21
Based on [the PEP](https://www.python.org/dev/peps/pep-0443/#user-api), it looks like you're missing data in your registration. It doesn't work with ambiguous types such as collections without types (or `Callable` on its own for example). — Maximilian Burszley, Oct 07 '20 at 17:28
@MaximilianBurszley Thanks for your effort investigating into this, but I'm afraid I cannot quite get the point. I tried to replace `Callable` with a more specific `Callable[[int], str]` but it doesn't work either. Nor does it work when I explicitly use `@notifyme.register(Callable)` or `@notifyme.register(Callable[[int], str])` :-( And if `Callable` cannot be used, what can I use to register the function type? — Yang Hanlin, Oct 07 '20 at 17:38
I found a workaround: `def _a(): pass` then `Function = type(_a)` and use that in the places expected for the annotation (`func: Function`). I did not test whether dispatch works properly, but the code runs. `type(lambda: None)` also returns the internal `function` class. — Maximilian Burszley, Oct 07 '20 at 17:43
@MaximilianBurszley Thank you for your workaround! It works well, but I cannot still understand why the built-in function type is not designed to be directly accessible. (P.S. Would you like to post your workaround as an answer? I'll upvote and accept it.) — Yang Hanlin, Oct 08 '20 at 01:07
I've run into a similar issue. It seems that singledispatch from python std lib requires a class as a type. It does not support things like 'list[int]'. I could solve this problem using external lib called multimethod. — user3357359, Aug 19 '22 at 11:57

lot · Answer 1 · 2021-08-25T14:36:06.833

5

https://docs.python.org/3/library/collections.abc.html#collections.abc.Callable

from collections import abc

@notifyme.register
def notifyme_default(func: abc.Callable) -> Callable:
    return notifyme_customized('The function is invoked.')(func)

edited Aug 25 '21 at 14:36

answered Dec 24 '20 at 02:45

lot

335
3
9

1

This solution seems to play better with tools like mypy. Mypy complained that the overwritten `function` object is not a valid type. No complaints with `collections.abc.Callable` – Ben Lindsay Jun 09 '21 at 15:28
1

This is a much better solution. Partials, callable class instances, and probably lots of other callable things are supported by this solution that get rejected by the accepted answer. – scnerd Aug 24 '21 at 21:35

score 2 · Accepted Answer · answered Oct 08 '20 at 01:21

I was unable to use typing.Callable with functools.singledispatch, but I did find a workaround by using a function class reference instead:

from functools import singledispatch
from typing import Callable

function = type(lambda: ())

@singledispatch
def notifyme(arg):
    return NotImplemented

@notifyme.register
def notifyme_customized(message: str) -> Callable[[Callable], Callable]:
    def decorator(func: Callable) -> Callable:
        def decorated_func(*args, **kwargs):
            print(str)
            return func(*args, **kwargs)
        return decorated_func
    return decorator

@notifyme.register
def notifyme_default(func: function) -> Callable:
    return notifyme_customized('The function is invoked.')(func)

How to register typing.Callable with Python @singledispatch?

Background

Problem

2 Answers2