0

I'm trying to figure out file hierarchy and local imports in python3. I've run into a problem with a file structure below:

└── project
    ├── run.py
    └── scripts
        ├── __init__.py
        ├── misc.py
        ├── script1.py
        └── script2.py

I use relative imports for all files in the "scripts" directory.

from misc import x, y, z

I use absolute imports in run.py.

from scripts.misc import a, b, c
from scripts.script1 import more_stuff

My goal is to have the python files independent and fully functional. However, when I attempt to execute run.py, I encounter an error.

from misc import x, y, z
ModuleNotFoundError: No module named 'misc'

I would expect relative paths to be relative to the original file and not adopt the path of the executed file. Can I fix this by modifying the imports or file structure?

It also appears I don't understand how __init__.py works. I want to re-use generic package names (like "scripts"). I had assumed that __init__.py files would be read immediately downstream relative to the executed file: if run.py is executed, only the scripts directory at the same level should be considered. I have found that a distant (unrelated?) "scripts" directory receives priority. Is this something that can be addressed with more careful absolute paths? Example below.

└── dir
    └── project1
    |   ├── run.py
    |   └── scripts
    |       ├── __init__.py
    |       └── settings.py
    └── subdir
        └── project2
            ├── run.py
            └── scripts
                ├── __init__.py
                └── settings.py

Executing run.py from "project1" will attempt to import the "scripts" directory from project2.

cannot import name 'variable' from 'scripts.settings' (/Users/.../dir/subdir/project2/scripts/settings.py)

Removing __init__.py from project2/scripts no longer produces the error when executing run.py from "project1".

Ghoti
  • 737
  • 4
  • 19

1 Answers1

1
  1. You are saying:

I use relative imports for all files in the "scripts" directory.

from misc import x, y, z

But this is not relative. For relative you need to have

from .misc import x, y, z
  1. To understand why the unrelated scripts is taking precedence, look on your sys.path and verify if indeed it comes before your scripts package. I can assume the is some leftover for ide path manipulation.
Lior Cohen
  • 5,570
  • 2
  • 14
  • 30
  • Problem 2 seems to be related to an odd iPython behavior. When I open a new iPython console, the first value in sys.path is replaced with the parent directory of the last executed script. I just tested this by making a new python file in my downloads directory. Now '/Users/username/Downloads' appears at the top of my sys.path in place of '/Users/username/anaconda/bin'. This behavior does not occur via command line when running: `ipython -c "import sys; print(sys.path)"`. I can probably write some dirty code to force the correct sys.path value back if I can't find a better solution. – Ghoti Dec 29 '20 at 20:37
  • Problem 1 looks like supplemental modules within a package aren't expected to be run standalone. Using if/else to determine how the imports should be called like https://stackoverflow.com/questions/16981921/relative-imports-in-python-3 should work. – Ghoti Dec 29 '20 at 21:01
  • Additional info on problem 2: Spyder iPython console has options for selecting the current working directory when producing a new console. Was encounter problems due to unawareness of behavior. – Ghoti Dec 29 '20 at 23:24