2

When I try to profile a class, I am unable to break down the time spent past each of my own methods and functions.

For example, using cProfile, I am able to do the following:

import numpy as np
import cProfile

class Test_Class:
    def __init__(self, length, width):
        self.length = length
        self.width = width
        self.populate()
        return None

    def populate(self):
        self.array = np.random.randint(0, 3, (self.length, self.width))
        return None

    def add(self, additional_array):
        self.array = np.add(self.array, additional_array)
        return None


def Test_Function():
    x, y = 3000, 2000
    test_array = np.random.randint(0, 2, (x, y))
    model_one = Test_Class(x, y)
    model_one.add(test_array)

cProfile.run("Test_Function()")

I am given this analysis:

         9 function calls in 0.214 seconds

   Ordered by: standard name

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.002    0.002    0.214    0.214 <string>:1(<module>)
        1    0.000    0.000    0.119    0.119 temp.py:11(populate)
        1    0.020    0.020    0.020    0.020 temp.py:15(add)
        1    0.002    0.002    0.212    0.212 temp.py:24(Test_Function)
        1    0.000    0.000    0.119    0.119 temp.py:5(__init__)
        1    0.000    0.000    0.214    0.214 {built-in method exec}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        2    0.190    0.095    0.190    0.095 {method 'randint' of 'mtrand.RandomState' objects}

Alternatively, using profilehooks I am able to do the following:

@profile(immediate=True)
def Test_Function():
    x, y = 3000, 2000
    test_array = np.random.randint(0, 2, (x, y))
    model_one = Test_Class(x, y)
    model_one.add(test_array)

Test_Function()

And I am given this analysis:

*** PROFILER RESULTS ***
Test_Function (C:/Users/mack/Desktop/temp2.py:19)
function called 1 times

         7 function calls in 0.205 seconds

   Ordered by: cumulative time, internal time, call count

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000    0.205    0.205 temp.py:19(Test_Function)
        2    0.185    0.093    0.185    0.093 {method 'randint' of 'mtrand.RandomState' objects}
        1    0.000    0.000    0.113    0.113 temp.py:5(__init__)
        1    0.000    0.000    0.113    0.113 temp.py:11(populate)
        1    0.020    0.020    0.020    0.020 temp.py:15(add)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        0    0.000             0.000          profile:0(profiler)

Neither technique will show how much time was spent in library functions. How can I get an analysis that shows me how much time was spent, for instance, in numpy.add()?

MackM
  • 2,906
  • 5
  • 31
  • 45
  • Related: http://stackoverflow.com/questions/4492860/increasing-the-depth-of-cprofiler-in-python-to-report-more-functions?rq=1 (No answers) – MackM May 14 '15 at 19:56
  • Related: http://stackoverflow.com/questions/4492535/profiling-a-method-of-a-class-in-python-using-cprofile?rq=1 (Answers do not profile in any more depth than above) – MackM May 14 '15 at 19:57
  • Also related: http://stackoverflow.com/q/20434042/3001761, http://stackoverflow.com/q/11005233/3001761. Do you have any particular reason to think that the answers will have changed? – jonrsharpe May 14 '15 at 19:59
  • @jonrsharpe do not believe those answers ever satisfied my question. I have tried using the profilehooks library as suggested without success, and my actual use case is too large to wrap each action in its own function. – MackM May 14 '15 at 20:17
  • That doesn't really answer *my* question! Why are you expecting this question to get different answers to the numerous previous similar questions? If you tried something without success, why not ask a question about that with a [minimal example](http://stackoverflow.com/help/mcve)? – jonrsharpe May 14 '15 at 20:18
  • @jonrsharpe Ah, I misunderstood. What part of the MCVE is missing? – MackM May 14 '15 at 20:22
  • ...the bit where you tried to use profilehooks? – jonrsharpe May 14 '15 at 20:23
  • @jonrsharpe I have added the the profilehooks trial to the question. – MackM May 14 '15 at 20:44
  • There's always the [*simple-dumb-but-works method*](http://stackoverflow.com/a/4299378/23771). If you take 10 stack samples, just look at them. If your code is on N of them, it takes N/10 of the time, more or less. If a built-in routine is on N or them, then it takes N/10 of the time. You might think you need more samples, but you'd be surprised what you can tell with just a few. – Mike Dunlavey May 15 '15 at 20:19
  • [yappi](https://code.google.com/p/yappi/) Also does not provide the desired behavior. – MackM May 18 '15 at 15:08

0 Answers0