Python modules as scripts

Question

I have a python program that relies on modules. The structure is the following.

program.py
modules/__init__.py
modules/mod1.py
modules/mod2.py
modules/utils.py

program.py includes all modules in the modules/ directory and the modules in modules/ include modules/utils.py via from .utils import *

The above works fine when running program.py. However, mod1.py and mod2.py also work as their own scripts, but running python mod1.py gives the error

from .utils import *
ModuleNotFoundError: No module named '__main__.utils'; '__main__' is not a package

Removing the . from the import makes the individual scripts work fine, but then the main program.py script can not find utils.

How can I make the import work for both the main program and the individual modules?

you could run as something like, e.g., `python3 -m venv`. the `-m` means run library module as script. — acushner, Apr 23 '20 at 22:51
Apologies, I already have one of those and have edited the question — Throckmorton, Apr 23 '20 at 23:06
You can not use relative imports in scripts. See [here](https://stackoverflow.com/questions/14132789/relative-imports-for-the-billionth-time) — modesitt, Apr 23 '20 at 23:16

score 2 · Accepted Answer · answered Apr 23 '20 at 23:21

You may find it better to have a single point of access to your program. If this is intended to be a command-line utility, you could create a wrapper for all the functionality using a commandline parser, like the builtin argparse, or an external library like click.

Using click, you could add a file called e.g. cli.py, and use that as your single point of access, and then use flags to access the different functions of your program. And using setuptools, you can create a clean entry point into your program.

So your file structure becomes:

|-__init__.py
|-program.py
|-cli.py
|-setuptools.py
|-modules
  |-mod1.py
  |-mod2.py
  |-utils.py

cli.py:

import click
import program

@click.group()
def cli():
    pass

@cli.command()
def program():
    program.run_the_program()

@cli.command()
def mod1():
    program.mod1.run_the_program()

@cli.command()
def mod2():
    program.mod2.run_the_program()

if __name__ == '__main__':
    cli()

And your setup.py:

from setuptools import setup

setup(
    name="your-program",
    version="1.0",
    py_modules=["program"],
    include_package_data=True,
    install_requires=["click"],
    entry_points="""
        [console_scripts]
        program=program:cli
    """,
)

Create a virtualenv in your directory, activate it, run setuptools pip install --editable, and then you should be able to run your program or the mod1 and mod2 sub-programs from the commandline using program program, program mod1 etc.

Python modules as scripts

1 Answers1