47

Problem

Most iPython "magic functions" work fine for me right off the bat: %hist, %time, %prun, etc. However, I noticed that %lprun could not be found with iPython as I'd installed it originally.

Attempt to Resolve

I then discovered that I should install the line_profiler module. I have installed this module, but still cannot seem to get the magic function to work correctly. If I attempt to call %lprun, iPython still cannot find the function. If I call it with the full name ( line_profiler.magic_lprun ), the function can be found, but I cannot get it to work at all. Below is an example of what I'd done (which is taken step by step from "Python for Data Analysis" book):

Success Using %prun

[In:]

def add_and_sum(x, y):
    added = x + y
    summed = added.sum(axis=1)
    return summed

x = randn(3000, 3000)
y = randn(3000, 3000)

add_and_sum(x, y)

With this I get a nice answer, as expected:

[Out:]

array([-23.6223074 , -10.08590736, -31.2957222 , ..., -14.17271747,
    63.84057725, -50.28469621])

And I can do the profiling magic function %prun:

[In:]

%prun add_and_sum(x, y)

[Out:]

6 function calls in 0.042 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
    1    0.020    0.020    0.029    0.029 <ipython-input-27-19f64f63ba0a>:1(add_and_sum)
    1    0.013    0.013    0.042    0.042 <string>:1(<module>)
    1    0.009    0.009    0.009    0.009 {method 'reduce' of 'numpy.ufunc' objects}
    1    0.000    0.000    0.009    0.009 _methods.py:16(_sum)
    1    0.000    0.000    0.009    0.009 {method 'sum' of 'numpy.ndarray' objects}
    1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}

Fail Using %lprun

But when I try %lprun, I cannot get anything:

[In:]

%lprun -f add_and_sum add_and_sum(x, y)

[Out:]

ERROR: Line magic function `%lprun` not found.

And if I try to call the function with its standard name, it also does not work:

[In:]

line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)

[Out:]

line_profiler.magic_lprun -f add_and_sum.test test.add_and_sum(x, y)
                                       ^
SyntaxError: invalid syntax

But the library has been imported properly, or at least this is what it says:

[In:]

line_profiler

[Out:]

<module 'line_profiler' from '/Users/<edit>/anaconda/lib/python2.7/site-packages/line_profiler-1.0b3-py2.7-macosx-10.5-x86_64.egg/line_profiler.pyc'>

[In:]

line_profiler.magic_lprun

[Out:]

<function line_profiler.magic_lprun>

It seems as if there is something extra that I am supposed to configure so that these new magic functions that I add can be identified as such. I could not find anything via a web search.

I am running Spyder as an IDE (still using iPython as the console), but I have also tried it directly with iPython and with iPython notebook. I have had no luck in any format.

Mike Williamson
  • 4,915
  • 14
  • 67
  • 104

2 Answers2

70

To make %lprun work, you need to load the extension into your session, using this command:

In [1]: %load_ext line_profiler

Check out this notebook to see some examples that use the magic.

Besides, if you are working with Spyder, there is also a third-party line_profiler plugin, which you can find here.

Carlos Cordoba
  • 33,273
  • 10
  • 95
  • 124
  • Awesome, thanks so much, Carlos!! This worked for me... I could not seem to find this information anywhere on the web. – Mike Williamson Nov 13 '13 at 20:31
  • 2
    +1 for the `line_profiler` plugin if you're using Spyder. I find it's the least troublesome way to run `line_profiler` and the ouput it gives is nice. – cd98 Aug 12 '14 at 15:10
  • 1
    sorry to say, but it doesn't work anymore (since IPython 5.) – Alleo Sep 10 '16 at 20:00
  • @Alleo, not my fault :-) The `line_profiler` extension hasn't been updated to work with IPython 5.0. Please see https://github.com/rkern/line_profiler/pull/65 for a fix – Carlos Cordoba Sep 11 '16 at 14:14
  • 1
    `%load_ext line_profiler` command works in IPython 5.0 with line_profiler version 2.0. – Steven C. Howell Nov 01 '16 at 19:28
30

You have two ways to make work %lprun, one solution is temporal, i.e., it lasts until you finish your ipython session, and the other one is permanent.

Temporal: (as Carlos Cordoba's answer)

After starting ipython run the following:

In [1]: %load_ext line_profiler

Permanent:

Add the following lines to ~/.ipython/profile_default/ipython_config.py:

c.TerminalIPythonApp.extensions = [
    'line_profiler',
]

If you don't have the file ~/.ipython/profile_default/ipython_config.py, you can create it by (see this for more info):

ipython profile create
lmiguelvargasf
  • 63,191
  • 45
  • 217
  • 228