Most efficient way of making an if-elif-elif-else statement when the else is done the most?

Question

I've got a in if-elif-elif-else statement in which 99% of the time, the else statement is executed:

if something == 'this':
    doThis()
elif something == 'that':
    doThat()
elif something == 'there':
    doThere()
else:
    doThisMostOfTheTime()

This construct is done a lot, but since it goes over every condition before it hits the else I have the feeling this is not very efficient, let alone Pythonic. On the other hand, it does need to know if any of those conditions are met, so it should test it anyway.

Does anybody know if and how this could be done more efficiently or is this simply the best possible way to do it?

Can you `sort` the things you are running your if/else... chain on, such that all the elements that one of the conditions will match for are at one end, and all the rest are at the other? If so, you could see if that is faster/more elegant or not. But remember, if there is no performance issue, it is too early to worry about optimization. — Patashu, Jun 18 '13 at 10:07
rel http://stackoverflow.com/questions/374239/why-doesnt-python-have-a-switch-statement — georg, Jun 18 '13 at 10:16
@Patashu - Unfortunately I do have an enormous performance issue. This program runs day and night analysing huge amounts of data. It's only that my C++ skills aren't up to par (and I love Python) that I'm not rewriting it.. — kramer65, Jun 18 '13 at 10:19
Is there something that the three special cases have in common? For example, you could do `if not something.startswith("th"): doThisMostOfTheTime()` and do another comparison in the `else` clause. — Tim Pietzcker, Jun 18 '13 at 10:20
@TimPietzcker - Unfortunately there isn't really anything they have in common. They can be 3 specific numbers or one specific string, if it is any other number or string (out of a 2000+ options), it has to do the else action. — kramer65, Jun 18 '13 at 10:22
@kramer65 If it's such a long chain of if/elif... it could be slow, but make sure to actually *profile* your code and start by optimising whatever part takes most time. — jorgeca, Jun 18 '13 at 14:25
Are these comparisons performed only once per value of `something`, or are similar comparisons performed multiple times on the same value? — Chris Pitman, Jun 18 '13 at 14:57

Aya · Accepted Answer · 2013-06-20T14:24:16.947

122

The code...

options.get(something, doThisMostOfTheTime)()

...looks like it ought to be faster, but it's actually slower than the if ... elif ... else construct, because it has to call a function, which can be a significant performance overhead in a tight loop.

Consider these examples...

1.py

something = 'something'

for i in xrange(1000000):
    if something == 'this':
        the_thing = 1
    elif something == 'that':
        the_thing = 2
    elif something == 'there':
        the_thing = 3
    else:
        the_thing = 4

2.py

something = 'something'
options = {'this': 1, 'that': 2, 'there': 3}

for i in xrange(1000000):
    the_thing = options.get(something, 4)

3.py

something = 'something'
options = {'this': 1, 'that': 2, 'there': 3}

for i in xrange(1000000):
    if something in options:
        the_thing = options[something]
    else:
        the_thing = 4

4.py

from collections import defaultdict

something = 'something'
options = defaultdict(lambda: 4, {'this': 1, 'that': 2, 'there': 3})

for i in xrange(1000000):
    the_thing = options[something]

...and note the amount of CPU time they use...

1.py: 160ms
2.py: 170ms
3.py: 110ms
4.py: 100ms

...using the user time from time(1).

Option #4 does have the additional memory overhead of adding a new item for every distinct key miss, so if you're expecting an unbounded number of distinct key misses, I'd go with option #3, which is still a significant improvement on the original construct.

edited Jun 20 '13 at 14:24

answered Jun 18 '13 at 10:29

Aya

39,884
6
55
55

3

does python have a switch statement? – nathan hayfield Jun 18 '13 at 15:08
ugh...well so far thats the only thing i've heard about python that i don't care for...guess there was bound to be something – nathan hayfield Jun 18 '13 at 18:28
2

-1 You say that using a `dict` is slower, but then your timings actually show that it is the second fastest option. – Marcin Jun 20 '13 at 15:02
12

@Marcin I'm saying that `dict.get()` is slower, which is `2.py` - the slowest of them all. – Aya Jun 20 '13 at 15:05
For the record, three and four are also dramatically faster than capturing the key error in a try/except construct. – Jeff Jun 15 '17 at 19:44
What about if the conditions are more complicated `3 > x > 1' – Moondra Jun 25 '20 at 21:46
1

3 and 4 are faster because performs a dictionary lookup with ìf x in ...` which is way faster than a function call i.e. `dict.get('blah', None)` , actually a set() lookup is even faster than a dict's lookup – Federico Baù Dec 15 '20 at 21:20

Ashwini Chaudhary · Answer 2 · 2013-06-18T14:17:04.923

87

I'd create a dictionary :

options = {'this': doThis,'that' :doThat, 'there':doThere}

Now use just:

options.get(something, doThisMostOfTheTime)()

If something is not found in the options dict then dict.get will return the default value doThisMostOfTheTime

Some timing comparisons:

Script:

from random import shuffle
def doThis():pass
def doThat():pass
def doThere():pass
def doSomethingElse():pass
options = {'this':doThis, 'that':doThat, 'there':doThere}
lis = range(10**4) + options.keys()*100
shuffle(lis)

def get():
    for x in lis:
        options.get(x, doSomethingElse)()

def key_in_dic():
    for x in lis:
        if x in options:
            options[x]()
        else:
            doSomethingElse()

def if_else():
    for x in lis:
        if x == 'this':
            doThis()
        elif x == 'that':
            doThat()
        elif x == 'there':
            doThere()
        else:
            doSomethingElse()

Results:

>>> from so import *
>>> %timeit get()
100 loops, best of 3: 5.06 ms per loop
>>> %timeit key_in_dic()
100 loops, best of 3: 3.55 ms per loop
>>> %timeit if_else()
100 loops, best of 3: 6.42 ms per loop

For 10**5 non-existent keys and 100 valid keys::

>>> %timeit get()
10 loops, best of 3: 84.4 ms per loop
>>> %timeit key_in_dic()
10 loops, best of 3: 50.4 ms per loop
>>> %timeit if_else()
10 loops, best of 3: 104 ms per loop

So, for a normal dictionary checking for the key using key in options is the most efficient way here:

if key in options:
   options[key]()
else:
   doSomethingElse()

edited Jun 18 '13 at 14:17

answered Jun 18 '13 at 10:07

Ashwini Chaudhary

244,495
58
464
504

`options = collections.defaultdict(lambda: doThisMostOfTheTime, {'this': doThis,'that' :doThat, 'there':doThere}); options[something]()` is marginally more efficient. – Aya Jun 18 '13 at 10:11
Cool idea, but not as readable. Also you'd probably want to separate the `options` dict to avoid rebuilding it, thereby moving part (but not all) of the logic far from the point of use. Still, nice trick! – Anders Johansson Jun 18 '13 at 10:17
7

do you _know_ whether this is more efficient? My guess is it's slower since it's doing a hash lookup rather than a simple conditional check or three. The question is about efficiency rather than compactness of code. – Bryan Oakley Jun 18 '13 at 11:22
2

@BryanOakley I've added some timing comparisons. – Ashwini Chaudhary Jun 18 '13 at 12:14
Finally my first `Populist` badge. – Ashwini Chaudhary Jun 19 '13 at 15:45
2

actually it should be more efficient to do ```try: options[key]() except KeyError: doSomeThingElse()``` (since with `if key in options: options[key]()` you're searching the dictionary twice for `key` – hardmooth Feb 18 '16 at 15:34
actually testing ```try: except:``` with 3.10 shows this to be 50% slower. – jwal Jun 21 '22 at 02:16

score 9 · Answer 3 · answered Jun 18 '13 at 16:02

Are you able to use pypy?

Keeping your original code but running it on pypy gives a 50x speed-up for me.

CPython:

matt$ python
Python 2.6.8 (unknown, Nov 26 2012, 10:25:03)
[GCC 4.2.1 Compatible Apple Clang 3.0 (tags/Apple/clang-211.12)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
>>> from timeit import timeit
>>> timeit("""
... if something == 'this': pass
... elif something == 'that': pass
... elif something == 'there': pass
... else: pass
... """, "something='foo'", number=10000000)
1.728302001953125

Pypy:

matt$ pypy
Python 2.7.3 (daf4a1b651e0, Dec 07 2012, 23:00:16)
[PyPy 2.0.0-beta1 with GCC 4.2.1] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And now for something completely different: ``a 10th of forever is 1h45''
>>>>
>>>> from timeit import timeit
>>>> timeit("""
.... if something == 'this': pass
.... elif something == 'that': pass
.... elif something == 'there': pass
.... else: pass
.... """, "something='foo'", number=10000000)
0.03306388854980469

Hi Foz. Thanks for the tip. In fact I am already using pypy (love it), but I still need speed improvements.. :) — kramer65, Jun 18 '13 at 16:11
Oh well! Before this I tried pre-computing a hash for 'this', 'that', and 'there' - and then comparing hash codes instead of strings. That turned out to be twice as slow as the original, so it looks like string comparisons are already pretty well optimised internally. — foz, Jun 18 '13 at 16:23

Arthur Julião · Answer 4 · 2020-06-29T12:31:52.027

Here an example of a if with dynamic conditions translated to a dictionary.

selector = {lambda d: datetime(2014, 12, 31) >= d : 'before2015',
            lambda d: datetime(2015, 1, 1) <= d < datetime(2016, 1, 1): 'year2015',
            lambda d: datetime(2016, 1, 1) <= d < datetime(2016, 12, 31): 'year2016'}

def select_by_date(date, selector=selector):
    selected = [selector[x] for x in selector if x(date)] or ['after2016']
    return selected[0]

It is a way, but may not be the most pythonic way to do it because is less readable for whom is not fluent in Python.

Pierre · Answer 5 · 2021-10-12T14:18:09.233

I tried with the match statement, introduced in python 3.10:

5.py

something = 'something'
for i in range(10000000):
    match something:
        case "this":
            the_thing = 1
        case "that":
            the_thing = 2
        case "there":
            the_thing = 3
        case _:
            the_thing = 4

Here are the results I get with 3.10.0:
1.py: 1.4s
2.py: 0.9s
3.py: 0.7s
4.py: 0.7s
5.py: 1.0s
I thought I would get something similar to 1.py but it is quite faster.

score 1 · Answer 6 · edited May 11 '20 at 01:21

1

People warn about exec for security reasons, but this is an ideal case for it.
It's an easy state machine.

Codes = {}
Codes [0] = compile('blah blah 0; nextcode = 1')
Codes [1] = compile('blah blah 1; nextcode = 2')
Codes [2] = compile('blah blah 2; nextcode = 0')

nextcode = 0
While True:
    exec(Codes[nextcode])

edited May 11 '20 at 01:21

SherylHohman

16,580
17
88
94

answered May 10 '20 at 21:13

user3319934

33
3

score 1 · Answer 7 · answered Oct 12 '21 at 14:54

you could immitate if-elif-else with switch-case type like by using dictionary and lambda function

For example:

x = 5
y = 5
operator = 'add'

def operation(operator, x, y): 
 return {
   'add': lambda: x+y,
   'sub': lambda: x-y,
   'mul': lambda: x*y,
   'div': lambda: x/y
 }.get(operator, lambda: None)()

result = operation(operator, x, y)
print(result)

score 0 · Answer 8 · answered Nov 23 '20 at 10:58

Recently I came across an approach alternative to "nested if else" which reduce running time of my function from 2.5hrs to ~2min..Baam! Let's begin:

Earlier Code
bin = lambda x:"Unknown" if x==0 else("High" if x>75 else("Medium" if x>50 and x<=75 else("Medium_Low" if x>25 and x<=50 else "Low")))

col.apply(bin) Time ~2.5hrs

Optimize Code

Define Dictionary alternative to nest if else

 def dict_function(*args):
'Pass in a list of tuples, which will be key/value pairs'
ret = {}
for k,v in args:
    for i in k:
        ret[i] = v
return ret
Dict = dict_function(([0],"Unknown"),(range(1,25),"Low"),(range(25,50),"Medium_Low"),(range(50,75),"Medium"),(range(75,100),"High"))

col.apply(lambda x:Dict[x])

dict_function make multiple key_value pairs for given range. Time~2mins

score 0 · Answer 9 · answered Dec 06 '20 at 16:36

I had the same problem recently, though not regarding performance, but I dislike the "API" of creating functions and manually adding them to a dict. I wanted an API similar to functools.singledispatch, but to dispatch based on values not on types. So ...

def value_dispatch(func):
    """value-dispatch function decorator.
    Transforms a function into a function, that dispatches its calls based on the
    value of the first argument.
    """
    funcname = getattr(func, '__name__')
    registry = {}

    def dispatch(arg):
        """return the function that matches the argument"""
        return registry.get(arg, func)

    def register(arg):
        def wrapper(func):
            """register a function"""
            registry[arg] = func
            return func
        return wrapper

    def wrapper(*args, **kwargs):
        if not args:
            raise ValueError(f'{funcname} requires at least 1 positional argument')
        return dispatch(args[0])(*args, **kwargs)

    wrapper.register = register
    wrapper.dispatch = dispatch
    wrapper.registry = registry
    return wrapper

Use like this:

@value_dispatch
def handle_something():
    print("default")

@handle_something.register(1)
def handle_one():
    print("one")

handle_something(1)
handle_something(2)

PS: I created a snippet on Gitlab for reference

Most efficient way of making an if-elif-elif-else statement when the else is done the most?

9 Answers9

5.py

Linked

Related