python modules hierarchy naming convention

Question

I'd like to have modules/packages structure like following:

/__init__.py
/mymodule.py
/mymodule/
/mymodule/__init__.py
/mymodule/submodule.py

And then use modules like:

import mymodule
import mymodule.submodule

But it seems like file "mymodule.py" conflicts with "mymodule" directory.

What's the correct naming convention here?

score 13 · Accepted Answer · answered Mar 06 '13 at 02:27

13

If you want to make a package, you have to understand how Python translates filenames to module names.

The file mymodule.py will be available as the mymodule, assuming the interpreter finds it in a directory in the Python search path. If you're on a case-insensitive filesystem, it might also be importable with different capitalization (but you should avoid using such system-dependent behavior).

A package is a directory with an __init__.py file in it. There's been some movement recently to allow packages without those files, but I'm going to ignore that less-common case for this answer. A package becomes a module inside Python, with its code coming from the __init__.py file. So the file mypackage/__init__.py can be imported as mypackage.

There's no meaning to an __init__.py file directly in the Python search path (well, I suppose you could import it an an __init__ module, but this is probably a bad idea).

So, for your situation, here's the appropriate filesystem layout:

toplevel/
    mymodule/
        __init__.py     # put code here for mymodule
        submodule.py    # put code here for mymodule.submodule

Only the toplevel folder should be in the Python search path.

answered Mar 06 '13 at 02:27

Blckknght

100,903
11
120
169

I've seen Python folk discourage putting significant code in `mymodule/__init__.py` though, eg. [Alex Martelli](http://stackoverflow.com/a/2361278/188535), which seems to be at odds with this. (Not that I necessarily agree with that...) – detly Mar 06 '13 at 02:35
2

That's true. A more popular style would be to put the actual code for `mymodule` into an undocumented submodule, then import the public API for it into `__init__.py`. – Blckknght Mar 06 '13 at 02:51
thanks for informative answer. It's pity that python needs such dirty hacks to do an obvious thing though. – Sergey Romanovsky Mar 06 '13 at 18:15
@SergeyRomanovsky - I wouldn't call it a hack - in the case you describe you'd have to look in *three* places just to see where a symbol was being defined (because `from mymodule import something` would look in `mymodule.__init__`, `mymodule.mymodule`, and `mymodule.mymodule.__init__`). Python 3.3 does revisit the package module though and updates it significantly. Well worth looking into. – Sean Vieira Mar 06 '13 at 18:24
@SeanVieira: You said "Python 3.3 does revisit the package module though and updates it significantly. Well worth looking into"? Would you elaborate, or not? – serv-inc Nov 18 '18 at 08:44
The module import specification was updated by PEP 302 to provide more hooks for importing, so you can hook into the way Python does module resolution and make it more sensible for your use cases if you really need to. See the Python 3.3+ docs for [`importlib`](https://docs.python.org/3/library/importlib.html#module-importlib) – Sean Vieira Nov 18 '18 at 17:45

Sean Vieira · Answer 2 · 2013-03-06T02:39:15.940

You are dealing with a package. The package structure you should have is:

/some-parent-directory # This needs to be on sys.path
    /mymodule  # This is not really a module - it's a package
        __init__.py  # import mymodule
        # init is loaded when you `import mymodule` or anything below it
        some.py  # import mymodule.some
        implementation.py  # import mymodule.implementation
        files.py  # import mymodule.files
        /submodule
            __init__.py  # import mymodule.submodule
            # init is loaded when you `import mymodule.submodule` or anything below it
            submodule_impl.py  # import mymodule.submodule.submodule_impl
            goes.py  # import mymodule.submodule.goes
            here.py  # import mymodule.submodule.here

As long as the parent directory is on sys.path you will be able to call import mymodule or from mymodule.submodule import something without issue.

If you want something to be available from the root level of a package (i. e. from mymodule import SomeItem or from a sub-package from mymodule.submodule import AnotherItem) then you can import it into the appropriate __init__.py file.

So, for example, let's say you wanted the class CustomClass defined in the submodule_impl.py module to be importable directly from submodule. Your submodule/__init__.py would have to contain the following:

from .submodule_impl import CustomClass

Then you would be able to import CustomClass directly from submodule (i. e. from mymodule.submodule import CustomClass)

I've seen your recommendations from the last few paragraphs a few times, it seems like that is the preferred way to move code out of \_\_init\_\_.py. My problem is, I don't actually want submodule_imp to be part of the public API. Like, that's an implementation detail. It shouldn't be conceptually possible to `import mymodule.submodule.submodule_imp`. Is there any way around this, or is it just the nature of Python that I shouldn't worry about it and I should trust library consumers to ignore things that are undocumented? — Tim M., Jan 21 '21 at 14:34
The latter - Python's philosophy is "we're all adults here" - unless you're taking extra steps people can just import random things from your modules anyway - if you only document the ways people _should_ interact with your package, that's generally what people will do. — Sean Vieira, Jan 25 '21 at 06:40

python modules hierarchy naming convention

2 Answers2

Linked