Accepted practice to provide access to a module-relative file in a setup.py application

Question

(Sorry for the wall of text, but I wanted to make sure all relevant details of my problem are included)

I am roughly following the application structure outlined here. I also have a logging.conf file, which I want my application to use in all cases:

• when calling my application with python -m myapp (I have a __main__.py for that)
• when doing cmd1, cmd2, cmd3, ... (I have some console_scripts in setup.py:entry_points)

The structure is roughly:

myapp/
├── build
├── dist
├── MANIFEST.in
├── myapp
│   ├── lib
│   │   ├── helpers.py
│   │   ├── __init__.py
│   │   └── startup.py
│   ├── logging.conf
│   ├── cmd
│   │   ├── cmd1.py
│   │   ├── cmd2.py
│   │   ├── cmd3.py
│   │   └── __init__.py
│   ├── __main__.py
│   ├── settings.py
│   └── tox.ini
├── README.md
├── requirements.txt
├── setup.cfg
├── setup.py
├── tests
└── venv

In one of my modules (lib/startup.py), used in all of my application entry points (__main__.py, cmd1, cmd2, cmd3, ...) I am (early) loading the logger configuration, with

logging.config.fileConfig('logging.conf')

In my MANIFEST.in, the logging.conf is installed. Everything seems to be installed correctly in the virtualenv when doing python setup.py install:

• the application is properly installed
• the logging.conf is copied
• the scripts (cmd1, ...) are created

And the application can be started, in all provided ways:

• python -m myapp
• cmd1, cmd2, cmd3

And I can also run it in my dev environment:

myapp/myapp $ python cmd/cmd1.py

But, whereas in the development environment logging.conf is found without problems (it is in the cwd), the installed module can not locate this file, and execution fails:

$ python -m cmd1

Traceback (most recent call last):
  File "/usr/lib/python3.4/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.4/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "myapp/venv/lib/python3.4/site-packages/myapp-0.5.0-py3.4.egg/myapp/__main__.py", line 5, in <module>
    from myapp.cmd.cmd1 import main
  File "myapp/venv/lib/python3.4/site-packages/myapp-0.5.0-py3.4.egg/myapp/cmd/cmd1.py", line 2, in <module>
    from myapp.lib.startup import start_trading
  File "myapp/venv/lib/python3.4/site-packages/myapp-0.5.0-py3.4.egg/myapp/lib/startup.py", line 10, in <module>
    logging.config.fileConfig('logging.conf')
  File "/usr/lib/python3.4/logging/config.py", line 76, in fileConfig
    formatters = _create_formatters(cp)
  File "/usr/lib/python3.4/logging/config.py", line 109, in _create_formatters
    flist = cp["formatters"]["keys"]
  File "/usr/lib/python3.4/configparser.py", line 937, in __getitem__
    raise KeyError(key)
KeyError: 'formatters'

How can I ensure that my installed application is able to find a module-relative file? I can not use absolute paths in my code, because I do not know the installation path ... Are there any recommended best practices for this?

Answer 1

DISCLAIMER. I'm not sure whether this approach is accepted practice. But it's definitely the simplest one.

You can use __file__ global variable to get relative path to the currently running *.py file. Having it you can calculate path to the resource file. Take a look at this answer for more details.

Given your example, you can calculate path to the logging.conf inside startup.py like this:

import os

# path to the logging.conf
config_path = os.path.join(os.path.dirname(__file__), '../logging.conf')
print(config_path)

# normalized path to the logging.conf (optionally) 
norm_config_path = os.path.normpath(config_path)
print(norm_config_path)