How to use timeit for multiple values

Question

I've written a code which solves the linear equation Ax=b using LU decomposition and I am asked to compute the running times of this for various dimensions of the matrix A.

I can compute the running time of the code, be it for given A and b or for random ones of various dimensions, but I'm not sure how to implement the part of computing it for every single dimension. I thought it might be a simple for loop, but nothing worked.

import timeit

setup = """
#function that solves the system
"""

repeats=5
result=timeit.timer('function', setup).timeit(number=repeats)/repeats

I also thought about using timer.repeat but it didn't really make sense, or I just couldn't understand how that would help me. Any tips would be greatly appreciated.

Welcome to StackOverflow! Is it possible you need the setup to create the matrix then the function to solve it? It is unclear whether you want help with `timeit` or an algorithm to generate matrices. — Tadhg McDonald-Jensen, Feb 17 '16 at 19:16
My setup creates the matrices and also solves the function, I do not know how to generate the running time for each matrix. For example, the setup takes random matrices with dimensions from 1 to 50 and for each of them it solves the system, and I can compute the running time for this code, but I do not know how to do it for each particular matrix of a different dimension. Does it make more sense? Since it's uni work, I am not allowed to post it online, so I cannot write the code here, sorry. — Drn004, Feb 17 '16 at 19:21
The way `timeit` works is that it runs the setup then times how long it takes to run the main code `number` times. To record running time for the solving part that needs to be the main statement. — Tadhg McDonald-Jensen, Feb 17 '16 at 19:37
Sorry, I don't really follow. Could you be a little more explicit? — Drn004, Feb 17 '16 at 19:59
run `help(timeit.Timer.timeit)` for the documentation, but it seems you are doing everything in the setup instead of using the main statement. After the matrix is made and solved in the setup what are is the `'function'` you are timing? It really seems like it needs to be the solving section. — Tadhg McDonald-Jensen, Feb 17 '16 at 20:07
Sorry, I can see that my example code is not very clear. The 'function' is only the solving part of the setup, not everything, so it only times how long it takes to run the solving section. — Drn004, Feb 17 '16 at 20:21

score 0 · Accepted Answer · edited May 23 '17 at 11:59

Just a note: you need to use timeit.Timer instead of timeit.timer or you will get AttributeError: module 'timeit' has no attribute 'timer' although you might as well use timeit.timeit which is useful for single use timing.

I'd recommend looking at How to use timeit module for efficient ways to use the timeit module, specifically using the from __main__ import syntax:

def function(stuff):
    NotImplemented

def setup():
    NotImplemented

setup_code = """
from __main__ import function, setup
data = setup() 
"""
func_code = "function(data)"

result = timeit.timeit(func_code,setup_code)

This way the setup function is called to generate the necessary data for the benchmark. Because it is a function it also has access to the global namespace in your module, so it can use information created in the main program in part of its code:

import timeit

def function(stuff):
    NotImplemented

def setup():
    print("the setup_data is:",setup_data)

setup_code = """
from __main__ import function, setup
data = setup() 
"""
func_code = "function(data)"

for i in range(4):
    setup_data = i
    result = timeit.timeit(func_code,setup_code)
    print("result is",result)

So just define setup_data as the data necessary for the setup (dimensions of matrix for example) before running timeit.timeit and you are good to go!

It makes sense to use repeat instead of timeit if you want to run the setup multiple times for the benchmarks but not every time.

For example lets say you have a sorting algorithm that varies in speed depending on how close to sorted the list is already. if you just use timeit the results could vary by a lot since after setup it will use the same list every time. You also wouldn't want to generate a list every pass since it would interfere with the actual benchmark.

So in that case it would make sense to use repeat to sort different lists without including the list creation in the benchmark, plus you can then use the entries to look at the max or min times and average etc.

How to use timeit for multiple values

1 Answers1