Performant way to partially apply in Python?

Question

I am looking for a way to partially apply functions in python that is simple to understand, readable, resusable and as little error prone to coder mistakes as possible. Most of all I want the style to be as performant as possible - less frames on the stack is nice, and less memory footprint for the partially applied functions is also desirable. I have considered 4 styles and written examples below:

import functools

def multiplier(m):
    def inner(x):
        return m * x

    return inner

def divide(n,d):
    return n/d

def divider(d):
    return functools.partial(divide,d=d)

times2 = multiplier(2)
print(times2(3))  # 6

by2 = divider(2)
print(by2(6)) # 3.0

by3 = functools.partial(divide,d=3)
print(by3(9)) # 3.0

by4 = lambda n: divide(n,4)
print(by4(12)) # 3.0

My analysis of them are:

times2 is a nested thing. I guess python makes a closure with the m bound, and everything is nice. The code is readable (I think) and simple to understand. No external libraries. This is the style I use today.

by2 has an explicit named function which makes it simple for the user. It uses functools, so it gives you extra imports. I like this style to some extent since it is transparent, and give me the option to use divide in other ways if I want to. Contrast this with inner which is not reachable.

by3 is like by2, but forces the reader of the code to be comfortable with functools.partial since they have it right in the face. what I like less is that PyCharm cant give my tooltips with what the arguments to functools.partial should be, since they are effectively arguments to by3. I have to know the signature of divide myself everytime I define some new partial application.

by4 is simple to type, since I can get autocompletion. It needs no import of functools. I think it looks non-pythonic though. Also, I always feel uncomfortable about scoping of variables / closures with lambdas work in python. Never sure how that behaves....

What is the logical difference between the styles and how does that affect memory and CPU?

Answer 1

The first way seems to be the most efficient. I tweaked your code so that all 4 functions compute exactly the same mathematical function:

import functools,timeit

def multiplier(m):
    def inner(x):
        return m * x

    return inner

def mult(x,m):
    return m*x

def multer(m):
    return functools.partial(mult,m=m)

f1 = multiplier(2)
f2 = multer(2)
f3 = functools.partial(mult,m=2)
f4 = lambda x: mult(x,2)

print(timeit.timeit('f1(10)',setup = 'from __main__ import f1'))
print(timeit.timeit('f2(10)',setup = 'from __main__ import f2'))
print(timeit.timeit('f3(10)',setup = 'from __main__ import f3'))
print(timeit.timeit('f4(10)',setup = 'from __main__ import f4'))

Typical output (on my machine):

0.08207898699999999
0.19439769299999998
0.20093803199999993
0.1442435820000001

The two functools.partial approaches are identical (since one of them is just a wrapper for the other), the first is twice as fast, and the last is somewhere in between (but closer to the first). There is a clear overhead in using functools over a straightforward closure. Since the closure approach is arguably more readable as well (and more flexible than the lambda which doesn't extend well to more complicated functions) I would just go with it.

Answer 2

Technically you're missing one other option as operator.mul does the same thing you're looking to do and you can just use functools.partial on it to get a default first argument without having to reinvent the wheel.

Not only is it the fastest option it also uses the least space compared to a custom function or a lambda statement. The fact that it's a partial is why it uses the same space as the others and I think that's the best route here.

from timeit import timeit
from functools import partial
from sys import getsizeof
from operator import mul

def multiplier(m):
    def inner(x):
        return m * x
    return inner

def mult(x,m):
    return m*x

def multer(m):
    return partial(mult,m=m)

f1 = multiplier(2)
f2 = multer(2)
f3 = partial(mult,m=2)
f4 = lambda n: mult(n,2)
f5 = partial(mul, 2)
from_main = 'from __main__ import {}'.format

print(timeit('f1(10)', from_main('f1')), getsizeof(f1))
print(timeit('f2(10)', from_main('f2')), getsizeof(f2))
print(timeit('f3(10)', from_main('f3')), getsizeof(f3))
print(timeit('f4(10)', from_main('f4')), getsizeof(f4))
print(timeit('f5(10)', from_main('f5')), getsizeof(f5))

Output

0.5278953390006791 144
1.0804575479996856 96
1.0762036349988193 96
0.9348237040030654 144
0.3904160970050725 96

This should answer your question as far as memory usage and speed.