Python project with C extensions - structure, imports and tests

Question

A project written in python with some C extensions (not using SWIG etc). I am trying to figure out how to structure my project so that:

1. imports work. Importing the shared objects
2. I don't need to change PYTHONPATH (tried figuring it out and failed).
3. In the future, distributing the project package will be easiest.

Current structure is, as suggested here:

Project\

    docs\  # mainly documentation, right?

    bin\   # empty

    setup.py # setup for the project, as suggested in the above link

    project\   
        __init__.py
        module.py

        tests\
             bla_test.py

    C\ # the package of C files
        file.c
        file.so
        other_c_stuff.c
        header.h
        setup.py # setup to compile the C files and create .so files
        build\ # contaisn a bunch of (hopefully) irrelevant stuf

It worked from PyDev but not from shell. Ideal answer would adress the following:

1. A suggested structure for the project.
2. How an import would be perfomed (from, say, the by modules in tests).
3. Should (can) I keep all the C files in a separate library?
4. The build of the C files is done in which of the setup.py files (should I post them here?)
5. Is it possible to automatically build when necessary? How?

I tried relative imports - they don't work for me for some reason. I saw the accepted answet to this question. He says - do whatever. But I can't get the imports to work. I read this answer, but have no clue what's all the stuff he has (and I don't have). The accepted answer doesn't help me because, again, the imports fail. This blog posts gives good advice but, again, the imports!

Answer 1

I don't want to go into detail for a general answer, since you linked to good ones already.

Some structure, which should work for you, could look like:

Project\

    build\ # directory used by setup.py

    docs\  # mainly documentation, right?

    setup.py # setup for the project, as suggested in the above link

    project\   
        __init__.py
        module.py

        c_package\
            __init__.py
            file.c
            file.so
            other_c_stuff.c
            header.h

        tests\
            __init__.py
            test_bla.py

So within the project package and its subpackages you can use relative imports, if you build the C Extensions inplace

python setup.py build_ext --inplace

or create a setup.cfg containing

[build_ext]
inplace=True

but only use this for development and don't release it, since installation will fail.

A build automation is possible, but I don't know of any other than calling setup.py directly, whenever the C sources have changed.