Measure time of a function with arguments in Python

Question

I am trying to measure the time of raw_queries(...), unsuccessfully so far. I found that I should use the timeit module. The problem is that I can't (= I don't know how) pass the arguments to the function from the environment.

Important note: Before calling raw_queries, we have to execute phase2() (environment initialization).

Side note: The code is in Python 3.

def raw_queries(queries, nlp):
    """ Submit queries without getting visual response """

    for q in queries:
        nlp.query(q)

def evaluate_queries(queries, nlp):
    """ Measure the time that the queries need to return their results """

    t = Timer("raw_queries(queries, nlp)", "?????")
    print(t.timeit())

def phase2():
    """ Load dictionary to memory and subsequently submit queries """

    # prepare Linguistic Processor to submit it the queries
    all_files = get_files()
    b = LinguisticProcessor(all_files)
    b.loadDictionary()

    # load the queries
    queries_file = 'queries.txt'
    queries = load_queries(queries_file)

if __name__ == '__main__':
    phase2()

Thanks for any help.

UPDATE: We can call phase2() using the second argument of Timer. The problem is that we need the arguments (queries, nlp) from the environment.

UPDATE: The best solution so far, with unutbu's help (only what has changed):

def evaluate_queries():
    """ Measure the time that the queries need to return their results """

    t = Timer("main.raw_queries(queries, nlp)", "import main;\
        (queries,nlp)=main.phase2()")

    sf = 'Execution time: {} ms'
    print(sf.format(t.timeit(number=1000)))


def phase2():
    ...

    return queries, b


def main():
    evaluate_queries()

if __name__ == '__main__':
    main()

Answer 1

First, never use the time module to time functions. It can easily lead to wrong conclusions. See timeit versus timing decorator for an example.

The easiest way to time a function call is to use IPython's %timeit command. There, you simply start an interactive IPython session, call phase2(), define queries, and then run

%timeit raw_queries(queries,nlp)

The second easiest way that I know to use timeit is to call it from the command-line:

python -mtimeit -s"import test; queries=test.phase2()" "test.raw_queries(queries)"

(In the command above, I assume the script is called test.py)

The idiom here is

python -mtimeit -s"SETUP_COMMANDS" "COMMAND_TO_BE_TIMED"

To be able to pass queries to the raw_queries function call, you have to define the queries variable. In the code you posted queries is defined in phase2(), but only locally. So to setup queries as a global variable, you need to do something like have phase2 return queries:

def phase2():
    ...
    return queries

If you don't want to mess up phase2 this way, create a dummy function:

def phase3():
    # Do stuff like phase2() but return queries
    return queries

Answer 2

Custom timer function may be a solution:

import time

def timer(fun,*args):
    start = time.time()
    ret = fun(*args)
    end = time.time()
    return (ret, end-start)

Using like this:

>>> from math import sin
>>> timer(sin, 0.5)
(0.47942553860420301, 6.9141387939453125e-06)

It means that sin returned 0.479... and it took 6.9e-6 seconds. Make sure your functions run long enough if you want to obtain reliable numbers (not like in the example above).

Answer 3

Normally, you would use timeit.

Examples are here and here.

Also Note:

By default, timeit() temporarily turns off garbage collection during the timing. The advantage of this approach is that it makes independent timings more comparable. This disadvantage is that GC may be an important component of the performance of the function being measured

Or you can write your own custom timer using the time module.

If you go with a custom timer, remember that you should use time.clock() on windows and time.time() on other platforms. (timeit chooses internally)

import sys
import time

# choose timer to use
if sys.platform.startswith('win'):
    default_timer = time.clock
else:
    default_timer = time.time

start = default_timer()
# do something
finish = default_timer()
elapsed = (finish - start)

Answer 4

I'm not sure about this, I've never used it, but from what I've read it should be something like this:

....
t = Timer("raw_queries(queries, nlp)", "from __main__ import raw_queries")
print t.timeit()

I took this from http://docs.python.org/library/timeit.html (if this helps).

Answer 5

You don't say so, but are you by any chance trying to make the code go faster? If so, I suggest you not focus in on a particular routine and try to time it. Even if you get a number, it won't really tell you what to fix. If you can pause the program under the IDE several times and examine it's state, including the call stack, it will tell you what is taking the time and why. Here is a link that gives a brief explanation of how and why it works.*

*When you follow the link, you may have to go the the bottom of the previous page of answers. SO is having trouble following a link to an answer.