Python: How to import a function from a package that depends on other functions in that package?

Question

Consider the following scenario:

test_setup/
├── __init__.py
├── helper.py
└── main.py

helper.py is:

def helper_func():
    print('helping')

main.py is:

from helper import helper_func

def main_func():
    helper_func()

if __name__ == '__main__':
    main_func()

__init__.py is:

from .main import main_func

I would like to be able to do two things:

1.run main from within the package. This works.

2.import main_func for use outside this package. This doesn't work. When called from the parent directory of test_setup, from test_setup import main_func yields:

In [1]: from test_setup import main_func
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-c395483f58c8> in <module>
----> 1 from test_setup import main_func

~/PycharmProjects/test_setup/__init__.py in <module>
      1 #from .helper import helper_func
----> 2 from .main import main_func

~/PycharmProjects/test_setup/main.py in <module>
----> 1 from helper import helper_func
      2 
      3 def main_func():
      4     helper_func()
      5 

ModuleNotFoundError: No module named 'helper'

If I change the first line in main.py to a relative import from .helper import helper_func that works, but fails when I just try to run from within the package (goal 1 above) yielding:

Traceback (most recent call last):
  File "/Users/xxx/PycharmProjects/test_setup/main.py", line 1, in <module>
    from .helper import helper_func
ImportError: attempted relative import with no known parent package

What's going on here, and how do I fix things to achieve goals 1 and 2? Trying to import helper_func in __init__.py didn't help either. The following failed as well:

In [1]: from test_setup.main import main_func
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-1-764832db473f> in <module>
----> 1 from test_setup.main import main_func

~/PycharmProjects/test_setup/__init__.py in <module>
----> 1 from .main import main_func

~/PycharmProjects/test_setup/main.py in <module>
----> 1 from helper import helper_func
      2 
      3 def main_func():
      4     helper_func()
      5 

ModuleNotFoundError: No module named 'helper'

chsws · Answer 1 · 2021-01-24T12:09:27.343

It should be: from test_setup.main import main_func to import the function directly. However, you can use it from the module.

import test_setup

test_setup.main_func()

On the edit, you have missed a . on from .helper import helper_func to indicate a relative import.

On further look at the goal 1 problem and this great post you should read, Guido himself advises against running scripts within module, rather than as a module.

The import docs say:

Note that relative imports are based on the name of the current module. Since the name of the main module is always "main", modules intended for use as the main module of a Python application must always use absolute imports.

This means scripts (things you would run independently from the package) should not be ran from within, if possible. If you need to keep the current structure, you can run the main.py file as a module using python -m test_setup.main which will work with relative imports.

Another option is adding the parent to test_setup folder to PYTHONPATH which will allow absolute imports such as test_setup.main.main_func from anywhere, although there are good threads suggesting you should use the proper development tools when building a package.

This approach works for running ```main_func``` from outside the package ```test_setup``` but fails when running ```main``` from within the package. — rhz, Jan 23 '21 at 23:33

just_a_passerby · Answer 2 · 2021-01-24T12:57:14.407

Don't know which version you are working with, but this may help you ...

__init__.py:

# 1. run main from within the package

# Python2
'''
>>> from main import main_func
main: from helper import helper_func
>>> main_func()
helping
'''

# Python3
'''
>>> from main import main_func
main: from helper import helper_func
>>> main_func()
helping
'''

# XXX: Notice that both of the above are the same
# However, note the differences below

# 2. import main_func for use outside this package

# Python2
'''
>>> from test_setup import main_func
test_setup.main: from helper import helper_func
test_setup: from main import main_func
>>> main_func()
helping
'''

# Python3
'''
>>> from test_setup import main_func
# Failed
ImportError:0 No module named 'main' in module test_setup
ImportError:0 No module named 'helper' in module test_setup.main
# Succeeded
test_setup.main: from .helper import helper_func
test_setup: from .main import main_func
>>> main_func()
helping
'''

try:
    from main import main_func
    print("%s: from main import main_func"%(__name__))
except ImportError as exc:
    print("ImportError:0 %s in module %s"%(exc,__name__))
    try:
        from .main import main_func
        print("%s: from .main import main_func"%(__name__))
    except ImportError as exc:
        print("ImportError:1 %s in module %s"%(exc,__name__))
        from test_setup.main import main_func
        print("%s: from test_setup.main import main_func"%(__name__))

main.py:

try:
    from helper import helper_func
    print("%s: from helper import helper_func"%(__name__))
except ImportError as exc:
    print("ImportError:0 %s in module %s"%(exc,__name__))
    try:
        from .helper import helper_func
        print("%s: from .helper import helper_func"%(__name__))
    except ImportError as exc:
        print("ImportError:1 %s in module %s"%(exc,__name__))
        from test_setup.helper import helper_func
        print("%s from test_setup.helper import helper_func"%(__name__))


def main_func():
    helper_func()

if __name__ == '__main__':
    main_func()

Edited for clarification:

__init__.py:

"""Init for main module"""

print("__init__.__doc__ = %s"%(__doc__))

# -----------------------------------------------------------------------------
from os import path
import sys

print("sys.executable = %s"%(sys.executable))
print("sys.argv[0] = %s"%(sys.argv[0]))

main_prg_path = path.abspath(path.dirname(__file__))
print("main_prg_path = %s"%(main_prg_path))

if getattr(sys, 'frozen', False):
    # When being bundled by pyinstaller, paths are different
    print("Running as pyinstaller bundle!", sys.argv[0])
    main_prg_path = path.abspath(path.dirname(sys.argv[0]))

sys.path.append(main_prg_path)
print("main_prg_path = %s"%(main_prg_path))
# Append other directories needed for main program
#sys.path.append(os.path.join(main_prg_path, 'utils'))

# -----------------------------------------------------------------------------
from main import main_func

main.py:

"""Main module"""

print("main.__doc__ = %s"%(__doc__))

# -----------------------------------------------------------------------------
# Running in python
# inside:  from main import main_func
# outside: from test_setup import main_func

# Running in terminal
# inside:  python -m main
# outside: python -m test_setup/main
# -----------------------------------------------------------------------------
from os import path
import sys

print("sys.executable = %s"%(sys.executable))
print("sys.argv[0] = %s"%(sys.argv[0]))

main_prg_path = path.abspath(path.dirname(__file__))
print("main_prg_path = %s"%(main_prg_path))

if getattr(sys, 'frozen', False):
    # When being bundled by pyinstaller, paths are different
    print("Running as pyinstaller bundle!", sys.argv[0])
    main_prg_path = path.abspath(path.dirname(sys.argv[0]))

sys.path.append(main_prg_path)
print("main_prg_path = %s"%(main_prg_path))
# Append other directories needed for main program
#sys.path.append(os.path.join(main_prg_path, 'utils'))

# -----------------------------------------------------------------------------
from helper import helper_func


def main_func():
    helper_func()

# -----------------------------------------------------------------------------
if __name__ == '__main__':
    print("running as main: %s"%(__name__))
    main_func()
else:
    print("running as file: %s"%(__file__))
    main_func()

helper.py:

"""Helper module"""

print("helper.__doc__ = %s"%(__doc__))

__author__ = "Your Name"
__author_email__ = "Your Email"
__version__ = "0.0.1"
__date__ = "24 Jan 2021"

def helper_func():
    print("Author = %s\nEmail = %s\nVersion = %s\nDate = %s"%(
        __author__, __author_email__, __version__, __date__))

@rhz EDIT: Added changes to show what is happening and simple usage example of sys.path.append

Pretty confusing. I'm using python 3.9. Using # outside: from test_setup import main_func — rhz, Jan 24 '21 at 22:29
Then change this: outside: `python -m test_setup/main` to `outside: python -m test_setup.main` (for python3) for terminal or use `from test_setup import main_func` for python interpreter. There's print statements to show what is happening for each way, but yes it does look confusing. When I am back at the PC I will update to explain what is going on. — just_a_passerby, Jan 25 '21 at 00:11

Python: How to import a function from a package that depends on other functions in that package?

2 Answers2