1

So would this par:

path = u'/a/b/c'
lam = lambda f: f.lower().startswith(u'distantlod')
par = functools.partial(lam, path)

and this par:

path = u'/a/b/c'
startsWith = path.lower().startswith
par = lambda: startsWith(u'distantlod')

be equivalent ? If not why ?

Mr_and_Mrs_D
  • 32,208
  • 39
  • 178
  • 361
  • 1
    Perhaps related: [What functionality does functools.partial offer that you can't get through lambdas?](http://stackoverflow.com/questions/3252228/python-why-is-functools-partial-necessary) – yuvi Oct 17 '15 at 11:45
  • @yuvi: It is related but does not answer this particular question - also related is: http://stackoverflow.com/q/17388438/281545 – Mr_and_Mrs_D Oct 17 '15 at 12:11
  • 2
    Could you clarify how you define 'equivalent'? – jamesc Oct 17 '15 at 12:34
  • 1
    @jamesc: in terms of performance (what inline is about). `par` is a function in both cases doing the same thing when called as par() - but in the second case `path.lower().startswith` is inlined in the call (saving us two dots). It seems python could do this in the first case - does it do it and if not why (or what am I missing) ? – Mr_and_Mrs_D Oct 17 '15 at 13:08
  • If you want to compare performance you can always [timeit](https://docs.python.org/2/library/timeit.html) – yuvi Oct 17 '15 at 13:11
  • @yuvi: would timeit answer why also ? ;) It's more of a language design question really (and a stylistic one maybe) – Mr_and_Mrs_D Oct 17 '15 at 13:15
  • If you're asking about usage difference as per styling and readability then I think the link I left answers that. If it's performance I believe the difference is negligible, but you can use timeit and test it out. What else are you asking for? – yuvi Oct 17 '15 at 13:18
  • I just tested it. While the second version is almost twice as fast, we're talking about a 200-250 nano-seconds difference, so yeah, pretty negligible. – yuvi Oct 17 '15 at 13:57

1 Answers1

0

Here's my experiment with cProfile to see if I can confirm @yuvi's measurements above.

CODE: par_profile.py

import cProfile as profile
import functools


path = u'/a/b/c'

lam = lambda f: f.lower().startswith(u'distantlod')
par = functools.partial(lam, path)

startsWith = path.lower().startswith
par2 = lambda: startsWith(u'distantlod')


if __name__ == '__main__':
    profile.run("for _ in range(1000000): par()")
    profile.run("for _ in range(1000000): par2()")

OUT

$ python par_profile.py 
         3000003 function calls in 0.536 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.148    0.148    0.536    0.536 <string>:1(<module>)
  1000000    0.242    0.000    0.388    0.000 par_profile.py:7(<lambda>)
        1    0.000    0.000    0.536    0.536 {built-in method exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  1000000    0.054    0.000    0.054    0.000 {method 'lower' of 'str' objects}
  1000000    0.092    0.000    0.092    0.000 {method 'startswith' of 'str' objects}


         2000003 function calls in 0.347 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.130    0.130    0.347    0.347 <string>:1(<module>)
  1000000    0.126    0.000    0.218    0.000 par_profile.py:11(<lambda>)
        1    0.000    0.000    0.347    0.347 {built-in method exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
  1000000    0.092    0.000    0.092    0.000 {method 'startswith' of 'str' objects}

Firstly it looks like these measurements on my machine tally with @yuvi's numbers:

  • par is about 540 nanoseconds
  • par2 is about 350 nanoseconds

So I agree par2 looks faster by about 200 ns.

It looks like if you're trying to compare lambda and partial it's not a fair test - par2 has one less call since it doesn't call lower, whereas par does.

To illustrate why, startsWith could be rewritten as:

lower_path = path.lower()
startsWith = lower_path.startswith

... so par2 is just a lambda that contains a single startswith whereas par is a partial that contains both lower and startswith calls.

Therefore they are not equivalent and par is slower as a result.

WHY?

The question is "why f.lower().startswith is not inlined - what prohibits the language from inlining it?"

Firstly, this Python itself does not prohibit this kind of inlining - it's the different Python implementations that makes the decisions, in the case of my tests above it's cpython 3.

Secondly, partial's job is not to inline functions, it just...

“freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature (doc)

Instead if you're looking at something that will do inlining in Python, then I'd check out something like Numba's jit or run some experiments with PyPy (please post back here if you find some interesting improvements).

If you're not able to find anything that'll do the inlining you're looking for, then maybe it's a good case for a new Python module!

jamesc
  • 5,852
  • 3
  • 32
  • 42