1

Example project structure:

|-setup.toml
|-README.md
|-tests/...
|-data/...
|-src/
   |-package1
   |-package2
         |-module1.py
   |-package3
         |-subpackage4
                |-module2.py

In reality, I have many more folders and files scattered around them and I need to be able to access $ROOT/data/... from most of them. Ideally, I wouldn't have to update numerous strings across the project every time I decide to move the file from one folder to another.

I can think of a number of solutions but none of them seem clean.

For example, I could pip install -e the package and have a module at the top level (just under src/) which would import os so I would just need to add ../data/ on top of that root. Then, I would import this module in all the other py files throughout the directory structure.

I think it would do the job but it feels very clumsy, surely there is a better way?

Looked through lots of documentation for setuptools, poetry etc. but none of the tutorials cover this trivial scenario. They are mostly concerned with dependencies and publishing to PyPi.

Is this still the way to do this in 2023? Python - Get path of root project structure

sinoroc
  • 18,409
  • 2
  • 39
  • 70
rudolfovic
  • 3,163
  • 2
  • 14
  • 38
  • Data files have to be part of an importable package in order to avoid a lot of inconveniences. I recommend you move `$ROOT/data` to `$ROOT/src/packagedata` for example, and make sure that this `packagedata` gets installed and is importable. – sinoroc Feb 07 '23 at 10:44
  • The code produces the data(sets) and there is a lot of data so it's not to be packaged with the code (it is also exclude from git etc.) – rudolfovic Feb 08 '23 at 09:59
  • 1
    Ah... Then I would recommend to place this data in one of the directories handled by something like [*`platformdirs`*](https://pypi.org/project/platformdirs/). – sinoroc Feb 08 '23 at 10:17
  • Thanks! That's good to know. And what would be a clean way to allow the user to customise their own folder instead? Also, would you still create a single `definitions.py` that will call platformdirs once and then import that everywhere else or would you invoke platformdirs everywhere directly? – rudolfovic Feb 08 '23 at 10:36
  • Uh, that's a different set of questions, but OK... ***1.*** You can have a configuration file in the `user_config_dir` that contains a field where the user has to write the path where to store the data, or you can make it a mandatory parameter to your CLI (or whatever application you have). ***2.*** I guess maybe I would write a helper function that I can reuse instead of calling `platformdirs` from multiple places throughout my code, but really it does not matter much. – sinoroc Feb 08 '23 at 10:53
  • I tried creating an `.env` file with `ROOT=$(pwd)` but `os.environ['ROOT']` returns `"$(pwd)"`. I also tried creating `definitions.py` at the end but even after running `poetry install` I am not able to import this module from the nested directories (even though VS code is able to auto-complete it). What am I missing.. – rudolfovic Feb 09 '23 at 14:40

0 Answers0