How to find the statistics and execution times of the code in python

Question

I am working on python and came across some concept of finding the statistics and execution time of the code

Suppose i had the following code

from time import gmtime, strftime
import timeit


def calculation():
     a = 2
     b = 3
     res = a + b
     return  res

if 'name' == 'main' :
    exec_time = timeit.timeit(calculation)
    print exec_time

result:

0.2561519145965576

So from the above code i am able to find the execution time of the code , but how to find the statistics of the code in python ?

Finally my intention is below points

1. How to find the statistics of the code in python
2. How to find the execution time of the entire code in python
3. What actually meant statistics of the code ?

Edited Code:

For example i had the above code in the file test.py

Now i had run the above file with the command below

python -m cProfile test.py

Result :

sh-4.2$ python -m cProfile test.py
         4 function calls in 0.001 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.001    0.001    0.001    0.001 test.py:1(<module>)
        1    0.000    0.000    0.000    0.000 timeit.py:105(Timer)
        1    0.001    0.001    0.001    0.001 timeit.py:53(<module>)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

So i need something like this when i run the above code, what i am trying is to write this functionality of printing statistics inside the file test.py instead of running the file with command python -m cProfile test.py from terminal.

At least i want to find the statistics and execution time of the function calculation() when the file runs because in real the function calculation has big functionality that performs some operation.

Answer 1

It appears what you are asking is how to the programmatic interface for the timeit module. That is documented here. You'll need a sample set to calculate statistics, such min, max, average, etc. from, which means running calculate numerous times through the repeat method of the Timeit class included in the timeit module.

For example:

timer = timeit.Timer(calculation)
results = timer.timeit(10000)

Answer 2

I think you are asking how to use cProfile from within the code. It actually turns out to be quite easy. cProfile.Profile has two undocumented methods, enable and disable, which can be used to start and stop the profiler. This means you can easily create a context manager or decorator. The following recipe implements both of these in one object, and includes a way of processing and printing the output with the pstat module.

import cProfile, pstats, functools

class profile:

    def __init__(self, filename=None):
        """
        If defined, output is written to *filename*, otherwise it
        is processed using *pstats* and printed to STDOUT.
        """
        self._filename = filename
        self._prof = cProfile.Profile()

    def __enter__(self):
        self._prof.enable()

    def __exit__(self, exc_type, exc_value, traceback):
        self._prof.disable()
        if self._filename is not None:
            self._prof.dump_stats(self._filename)
        else:
            stats = pstats.Stats(self._prof)
            self.print_stats(stats)

    def print_stats(self, stats):
        """
        This method processes the stats and prints them.
        """
        stats.strip_dirs().sort_stats('cumulative').print_stats(20)

    def __call__(self, func):
        self._func = func
        @functools.wraps(func)
        def wrapper(*args, **kwargs):
            with self:
                return func(*args, **kwargs)
        return wrapper

So usage is:

@profile()
def calculation():
    a = 2
    b = 3
    res = a + b
    return  res

calculation()

or

with profile('output_file.pstat'):
    calculation()

You can change print_stats as necessary to show the output you want.

Answer 3

There is another approach to finding the aggregation stats of the code. You can find the mean, median, standard deviation, total time spent, and number of times called using this library.

The library is called codetiming from realpython.

A sample use case:

from codetiming import Timer
from humanfriendly import format_timespan
from loguru import logger


@Timer(name="my_func", text=lambda secs: f"my_func elapsed time: {format_timespan(secs)}")
def my_func():
    ...

def get_aggregated_timings(cls):
    timed_function = "my_func"
    logger.info(
        f"\n{timed_function} count: {Timer.timers.count(timed_function)}\n"
        f"total: {Timer.timers.total(timed_function)}\n"
        f"max: {Timer.timers.max(timed_function)}\n"
        f"min: {Timer.timers.min(timed_function)}\n"
        f"mean: {Timer.timers.mean(timed_function)}\n"
        f"standard deviation: {Timer.timers.stdev(timed_function)}\n"
        f"median: {Timer.timers.median(timed_function)}\n"
    )
    Timer.timers.clear()  # clears all the timer data