Python if vs try-except

Question

I was wondering why the try-except is slower than the if in the program below.

def tryway():
    try:
        while True:
            alist.pop()
    except IndexError:
        pass

def ifway():
    while True:
        if alist == []: 
            break
        else:
            alist.pop()
if __name__=='__main__':
    from timeit import Timer
    alist = range(1000)
    print "Testing Try"
    tr = Timer("tryway()","from __main__ import tryway")
    print tr.timeit()
    print "Testing If"
    ir = Timer("ifway()","from __main__ import ifway")
    print ir.timeit()

The results I get are interesting.

Testing Try
2.91111302376
Testing If
0.30621099472

Can anyone shed some light why the try is so much slower?

Answer 1

You're setting alist only once. The first call to "tryway" clears it, then every successive call does nothing.

def tryway():
    alist = range(1000)
    try:
        while True:
            alist.pop()
    except IndexError:
        pass

def ifway():
    alist = range(1000)
    while True:
        if alist == []:
            break
        else:
            alist.pop()
if __name__=='__main__':
    from timeit import Timer
    print "Testing Try"
    tr = Timer("tryway()","from __main__ import tryway")
    print tr.timeit(10000)
    print "Testing If"
    ir = Timer("ifway()","from __main__ import ifway")
    print ir.timeit(10000)

>>> Testing Try
>>> 2.09539294243
>>> Testing If
>>> 2.84440898895

Answer 2

Exception handling is generally slow in most languages. Most compilers, interpreters and VMs (that support exception handling) treat exceptions (the language idiom) as exceptions (uncommon). Performance optimization involves trade-offs and making exceptions fast would typically mean other areas of the language would suffer (either in performance or simplicity of design).

At a more technical level, exceptions generally mean that the VM/interpretter (or the runtime execution library) has to save a bunch of state and begin pulling off all the state on the function call stack (called unwinding) up until the point where a valid catch (except) is found.

Or looking at it from a different viewpoint, the program stops running when an exception occurs and a "debugger" takes over. This debugger searches back through the stack (calling function data) for a catch that matches the exception. If it finds one, it cleans things up and returns control to the program at that point. If it doesn't find one then it returns control to the user (perhaps in the form of an interactive debugger or python REPL).

Answer 3

If you are really interested in speed, both of your contestants could do with losing some weight.

while True: is slower than while 1: -- True is a global "variable" which is loaded and tested; 1 is a constant and the compiler does the test and emits an unconditional jump.

while True: is redundant in ifway. Fold the while/if/break together: while alist != []:

while alist != []: is a slow way of writing while alist:

Try this:

def tryway2():
    alist = range(1000)
    try:
        while 1:
            alist.pop()
    except IndexError:
        pass

def ifway2():
    alist = range(1000)
    while alist:
        alist.pop()

`

Answer 4

There is still faster way iterating with for, though sometimes we want list to physically shirink so we know how many are left. Then alist should be parameter to the generator. (John is also right for while alist:) I put the function to be a generator and used list(ifway()) etc. so the values are actualy used out of function (even not used):

def tryway():
    alist = range(1000)
    try:
        while True:
            yield alist.pop()
    except IndexError:
        pass

def whileway():
    alist = range(1000)
    while alist:
         yield alist.pop()

def forway():
    alist = range(1000)
    for item in alist:
         yield item

if __name__=='__main__':
    from timeit import Timer
    print "Testing Try"
    tr = Timer("list(tryway())","from __main__ import tryway")
    print tr.timeit(10000)
    print "Testing while"
    ir = Timer("list(whileway())","from __main__ import whileway")
    print ir.timeit(10000)
    print "Testing for"
    ir = Timer("list(forway())","from __main__ import forway")
    print ir.timeit(10000)

J:\test>speedtest4.py
Testing Try
6.52174983133
Testing while
5.08004508953
Testing for
2.14167694497

Answer 5

Not sure but I think it's something like this: the while true follow the normal instruction line which means the processor can pipeline and do all sorts of nice things. Exceptions jump straight through all that so the VM need to handle it specially, and that takes time.

Answer 6

defensive programming requires that one test for conditions which are rare and/or abnormal, some of which during the course of a year or many years will not occur, thus in these circumstances perhaps try-except may be justified.

Answer 7

Just thought to toss this into the mix: I tried the following script below which seems to suggest that handling an exception is slower than handling an else statement:

import time

n = 10000000
l = range(0, n)
t0 = time.time()
for i in l:
    try:
        i[0]
    except:
        pass
t1 = time.time()
for i in l:
    if type(i) == list():
        print(i)
    else:
        pass
t2 = time.time()
print(t1-t0)
print(t2-t1)

gives:

5.5908801555633545
3.512694835662842

So, (even though I know someone will likely comment upon the use of time rather than timeit), there appears to be a ~60% slow down using try/except in loops. So, perhaps better to go with if/else when going through a for loop of several billion items.