IPython autoreload changes in subdirectory

Question

I launch IPython from the main folder /project. Now if I make changes in the file /project/tests/some_module.py, the changes fail to be autoreloaded in IPython. Also, I get the following message after I save the changes and want to run some other script in the prompt:

[autoreload of some_module failed: Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/IPython/extensions/autoreload.py", line 229, in check
    superreload(m, reload, self.old_objects)
ImportError: No module named some_module]

It seems it detects changes were made inside folder /tests but it cannot import it. Can someone help me with this?

Edit: For better clarification: I launch IPython from the terminal in the main folder. In this folder, I have another folder tests. Inside tests I have two python files:

some_module.py:

def hello_world():
  print "Hello World!!!"

use_some_module.py:

from some_module import hello_world

hello_world()

Once I launched IPython, further changes in some_module.py won't be loaded in IPython. For example, if I add a second print "Hello Earth!!!" in the definition of hello_world(), and run run tests/use_some_module.py, I get the error message shown above, and will only get the "Hello World!!!" print.

Edit2: I would like a solution where I don't need to either change the working directory, or adding any search paths manually. I want it to be loaded automatically with autoreload.

How do yo import your module the first time? Do you change the working directory between first import and reload? — Mike Müller, Nov 01 '15 at 11:27
@michael from your Edit2, the error `ImportError: No module named some_module` means, the `superreload` function of `autoreload` module couldn't find `some_module` because `some_module` isn't in search path. — Nizam Mohamed, Nov 02 '15 at 17:49
@NizamMohamed If I exit ipython and relaunch it again in the same parent directory, the changes get updated. — Michael, Nov 03 '15 at 14:08
@NizamMohamed So, what I want is for it to do it automatically every time there is a change in `tests/`. I have `autoreload 2` added in my ipython config file already, and it does the job for changes in the same directory I launched ipython in, but it isn't able to do the job for changes in the subdirectory. I want a solution that I can add to my ipython config file so that it does it automatically. — Michael, Nov 03 '15 at 14:17

score 2 · Answer 1 · edited May 23 '17 at 10:28

If all you need is to reload a changed file in Python just do the following:

from main import some_module
....
reload(some_module)

But if your reload purposes are really eager, you can do the following (taken from this question):

%load_ext autoreload
%autoreload 2

The previous code will reload all changed modules every time before executing a new line.

NOTE: You can also check dreload which does a recursive reload of a module, and %run which allows you to run any python script and load all of its data directly into the interactive namespace.

Hope it helps,

Nizam Mohamed · Answer 2 · 2015-11-03T16:13:59.000

Module /project/tests/some_module.py must be importable from /project. It means /project/tests must be in sys.path.

Either change to /project/tests and %run use_some_module.py or do sys.path.insert(0,tests) to insert tests into module search path.

If some_module is not in search path, IPython's autoreload never gonna find it.

Another way to make it work is to make the tests a package by creating __init__.py in it. And use relative import in use_some_module as from .some_module import hello_world. Now do these from IPython prompt,

In [1]: load_ext autoreload

In [2]: %autoreload 2

In [3]: from tests import some_module

In [4]: run -m tests.use_some_module
Hello World!!!
In [5]: ed tests/some_module.py
"tests/some_module.py" 3L, 69C written
 done. Executing edited code...
In [6]: run -m tests.use_some_module
Hello World!!!
Hello World!!!

That's you have to run use_some_module module in the tests package as a script.

The first time you do %run tests/use_some_module.py, which is run as if you were running python tests/use_some_module.py. So /project/tests which the script is in, is automatically included in sys.path. That's why from some_module import hello_world in use_some_module succeeds. Additionally, after the run, objects in use_some_module's global namespace are available in IPython session.

But when you change tests/some_module, it has to be loaded again to see the changes. To reload it manually, it has to be imported first. Now import should succeed because use_some_module imported it first when it was run, some_module is in sys.modules. But reload to succeed, some_moudle has to be in search path. So, even manual reload would fail let alone autoreload.

In [2]: #before %run  
In [3]: 'some_module' in globals()
Out[3]: False

In [6]: 'some_module' in sys.modules
Out[6]: False

In [7]: 'hello_world' in globals()
Out[7]: False

In [8]: run tests/use_some_module.py
Hello World!!!

In [9]: 'some_module' in globals()
Out[9]: False

In [11]: 'some_module' in sys.modules
Out[11]: True

In [12]: 'hello_world' in globals()
Out[12]: True
In [13]: import some_module

In [14]: some_module = reload(some_module)
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-14-47e3c0de23e1> in <module>()
----> 1 some_module = reload(some_module)

ImportError: No module named some_module

Another solution is to start IPython as PYTHONPATH=tests ipython to include tests directory in sys.path.
Or set this c.InteractiveShellApp.exec_lines = ['import sys','sys.path.insert(0,"tests")'] in ipython_config.py.
.

Thanks for the reply. I'm kind of new in Python, so I am not sure if I understand it fully. Why is it that autoreload is not clever enough to look up in the same path as the file I'm running? (ie. `tests/`). I mean, it does it in the first `run`, but later it doesn't anymore. BTW: I didn't downvoted you. — Michael, Nov 04 '15 at 13:55
look at [search path](https://docs.python.org/2/tutorial/modules.html#the-module-search-path) for more info. I've already answered you question. The first run imports `hello_world` from `some_module`. Subsequent runs from the same session won't re-import an already imported module. That's why we need `reload`. But the `reload` can't see a module which in not in search path. You have to tell it to look up in some paths. `autoreload` is automatic version of `reload`. `autoreload` and `%run` are two different things. `%run` is IPython's mechanism to run an external module. — Nizam Mohamed, Nov 04 '15 at 14:22
Can't `reload` automatically include `tests/` in the `sys.path` like `%run` does? — Michael, Nov 04 '15 at 15:35
`%run` uses `execfile`. Check the syntax of `execfile` and `reload`. `execfile` uses path name `reload` uses an already loaded module name. — Nizam Mohamed, Nov 04 '15 at 16:19

Mike Müller · Answer 3 · 2015-11-01T12:16:29.603

0

Try changing your file use_some_module.py to:

import some_module

some_module.hello_world()

Reload with from import usually does not work.

This works for me:

%autoreload 1
%aimport use_some_module
%aimport some_module

edited Nov 01 '15 at 12:16

answered Nov 01 '15 at 11:58

Mike Müller

82,630
20
166
161

1

No luck. Are you able to replicate my problem? – Michael Nov 01 '15 at 12:00

IPython autoreload changes in subdirectory

3 Answers3

Linked