Module/Project structure in Python

Question

I'm trying to build a simple project in Python,however I can't seem to wrap my head around how it's done despite reading the documentations over and over and reading example code.

I'm semi-new to Python, having a background of only single-file scripting. However I made numerous Node.js and Java projects with multiple folders and packages. In Python However I can't seem to grasp the project structure and workflow mechanisms- init.py, virtualenvs, setuptools etc. it all seems very alien and unintuitive to me, hurting my overall productivity(ironically Python is supposed to be a language aimed at productivity, right?)

My project has this example structure:

package_test  (package)
    |
    |------- __init__.py
    |------- main.py (entry point)
    |
    |------- /lib (modules)
    |           |
    |           |----- __init__.py
    |           |----- lib.py
    |
    |------- /Tests
    |           |
    |           |----- __init__.py
    |           |----- test.py

in main.py:

 from lib import lib

 lib.libtest()

in lib.py:

 def libtest():
     print("libtest")

in test.py:

 from lib import lib

 lib.libtest()

Running main.py works, However when running test.py it cannot find the module. I have tried solutions such as appending to sys.path, putting '..' before lib, and more- none have worked.

This is just an example, but I wish to develop more complex projects in Python with multiple subfolders in the future(I think Python has some cool features and nice libraries), yet this problem keeps bothering me. I never had to think about these issues when developing in Java or in Node, not to mention stuff such as virtualenv etc.

Thank you in Advance

Not an anwer to you question actually but virtualenv and setuptools are totally unrelated to your problem. And believe me, virtualenv __is__ a very cool feature - actually one you cannot hope to live without when you have more than on single project to maintain. — bruno desthuilliers, Jul 06 '17 at 11:48
Also you may want to read this (at least the first two answers) https://stackoverflow.com/questions/11536764/how-to-fix-attempted-relative-import-in-non-package-even-with-init-py — bruno desthuilliers, Jul 06 '17 at 11:50

BoboDarph · Answer 1 · 2017-07-06T12:16:01.747

0

Your test.py needs to know lib.py is one level above it and inside the lib folder. So all you should need to do is append the path of your lib folder before running the import of lib.py into test.py. Something like

import sys
sys.path.append("..//lib")
import lib

Please note that the answer above assumes you run on Windows. A more comprehensive approach would be to replace the windows paths with proper os.path syntax. Also another assumption made by this code is that you will run your test.py from it's folder, not with an absolute/relative path.

import sys
import os.path
sys.path.append(os.path.join(os.path.pardir, 'lib'))
import lib

If you want to run your test.py with a path, your code could look like this:

import sys
import os.path
# os.path.abspath(__file__) gets the absolute path of the current python scrip
# os.path.dirname gets the absolute path of whatever file you ask it to (minus the filename)
# The line below appends the full path of your lib folder to the current script's path
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), os.path.pardir, 'lib'))
# This should allow test.py to import libtest regardless of where your run it from. Assuming your provided project structure.
from lib import libtest
libtest()

edited Jul 06 '17 at 12:16

answered Jul 06 '17 at 09:37

BoboDarph

2,751
1
10
15

Are you still trying to import lib from lib (which, btw, doesnt exist) or did you give my code a try? – BoboDarph Jul 06 '17 at 09:44
I tried your code with os-agnostic relative path, still same result. I use Linux btw – ClovisIRex Jul 06 '17 at 09:46
Is the code in my answer in you test.py file? Also please output the full stack of the ImportError – BoboDarph Jul 06 '17 at 09:50
Yes, in test.py. My overall aim is to build cross-platform projects with a unified structure. All I want is to be able to deploy them the same on Windows,Linux etc like I do in Java and Node. Yet Python behaves differently from my expectations – ClovisIRex Jul 06 '17 at 09:52
Are you trying to run test.py or main.py which probably tries to import test.py? – BoboDarph Jul 06 '17 at 09:54
I run test.py, simulating a unit test in a future project – ClovisIRex Jul 06 '17 at 09:55
Can you confirm that your lib folder is actually called "lib" and your pyfile inside it is called "lib.py"? Also could you please post the full stack of the importError you get? – BoboDarph Jul 06 '17 at 09:58
I renamed the files just to be sure, it seems it works obly if I run test.py from the terminal when I cd'd into 'tests' folder, not when i run it from outside of tests using the full path. Very strange.. – ClovisIRex Jul 06 '17 at 11:54
Not strange at all. You are using relative paths. If you run the script script from one level above, there is no folder named "lib" above your package_test folder (hopefully) so your import will fail. See https://docs.python.org/2/tutorial/modules.html#the-module-search-path. I'll ammend the example code to fix this problem – BoboDarph Jul 06 '17 at 12:08

Module/Project structure in Python

1 Answers1