synthesized submodule: from A import B (ok) vs. import A.B (error)?

Question

I have a module modA, which contains a synthesized submodule modB (created with PyModule_New); now importing the module:

from modA import modB it is OK
import modA.modB fails.

What am I missing?

modA.cpp (using boost::python, but it would be very likely the same with pure c-API of python):

#include<boost/python.hpp>
namespace py=boost::python;

BOOST_PYTHON_MODULE(modA){
   py::object modB=py::object(py::handle<>(PyModule_New("modB")));
   modB.attr("__file__")="<synthetic>";
   py::scope().attr("modB")=modB;
};

compile with (g++ instead of clang++ works the same)

clang++ -o modA.so modA.cpp -fPIC -shared  -lboost_python `pkg-config python --cflags --libs`

test.py:

import sys
sys.path.append('.')
from modA import modB
import modA.modB

python test.py (note the first import is just fine, it is the second one which fails):

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    import modA.modB
ImportError: No module named modB

I just did that, thanks for suggestion. Have to wait 2 days to accept it :-) — eudoxos, Jun 16 '12 at 12:39

score 1 · Accepted Answer · edited May 23 '17 at 12:29

Thanks to this answer, I found the solution, which consists in sys.modules['modA.modB']=modB, but written in c++ in the module initialization function:

#include<boost/python.hpp>
namespace py=boost::python;

BOOST_PYTHON_MODULE(modA){
   py::object modB=py::object(py::handle<>(PyModule_New("modA.modB")));
   modB.attr("__file__")="<synthetic>";
   py::scope().attr("modB")=modB;

   // this was the missing piece: sys.modules['modA.modB']=modB
   py::extract<py::dict>(py::getattr(py::import("sys"),"modules"))()["modA.modB"]=modB;
};

score 1 · Answer 2 · edited Aug 26 '18 at 03:06

Here is the pattern I use, hopefully it improves previous answers...

module.h:

...
#define STR(str) #str
#define MAKE_SUBMODULE(mod, submod) object submod##_module(handle<>(borrowed(PyImport_AddModule(STR(mod.submod)))));\
scope().attr(STR(submod)) = submod##_module;\
scope submod##_scope = submod##_module;\
...
export_submodule();
...

module.cpp:

BOOST_PYTHON_MODULE(modulename)
{
    export_submodule();
}

submodule.cpp:

export_submodule()
{
    // create submodule
    MAKE_SUBMODULE(modulename, submodulename)

    // all code below is now in submodule's scope
    ...
}

The "generated" code looks like this:

object submodulename_module(handle<>(borrowed(PyImport_AddModule("modulename.submodulename"))));
scope().attr("submodulename") = submodulename_module;
scope submodulename_scope = submodulename_module;

It looks similar to eudoxos answer, but the differencs seems to be in detail: Instead of PyImportNew() I use PyImportAdd(). Therefor I don't need the last line to get the from module.submodule import * statement working.

score 0 · Answer 3 · answered Jun 16 '12 at 12:25

0

modB is not a file (i.e. a module), but some object in modA, hence it cannot be imported.

You may want to see Python's modules docs.

answered Jun 16 '12 at 12:25

Vidul

10,128
2
18
20

`modB` is a module, it just happens to be created on the fly. – eudoxos Jun 16 '12 at 12:26
I don't understand. Please specify - is `modB` a file or not? – Vidul Jun 16 '12 at 12:29
No, `modB` is a module object which is being created at runtime. – eudoxos Jun 16 '12 at 12:30
`imp.find_module` only searches files, according to documentation. Sorry for confusion, though, I found the solution meanwhile. – eudoxos Jun 16 '12 at 12:37

synthesized submodule: from A import B (ok) vs. import A.B (error)?

3 Answers3