4

I have a python package named pytools. It contains a cython-based submodule nms.

When I install the root package pytools with sudo python -H setup.py, the root package seems to be installed properly.

But the installation didn't copy compiled nms.so to /usr/local/lib/python2.7/dist-packages/pytools/nms/.

And When I import pytools in ipython, an error encountered:

ImportError: cannot import name nms

If I manually copy the pytools/nms/nms.so to /usr/local/lib/python2.7/dist-packages/pytools/nms/, the problem is solved.

Here is my setup.py of the root package:

import os
import numpy
from distutils.core import setup, Extension
from Cython.Build import cythonize

exec(open('pytools/version.py').read())
exts = [Extension(name='nms',
                  sources=["_nms.pyx", "nms.c"],
                  include_dirs=[numpy.get_include()])
        ]
setup(name='pytools',
  version=__version__,
  description='python tools',
  url='http://kaiz.xyz/pytools',
  author_email='zhaok1206@gmail.com',
  license='MIT',
  packages=['pytools', 'pytools.nms'],
  #packages=['pytools'],
  zip_safe=False
)

And setup.py of sub-package nms:

from distutils.core import setup, Extension
import numpy
from Cython.Distutils import build_ext
setup(
    cmdclass={'build_ext': build_ext},
    ext_modules=[Extension("nms",
    sources=["_nms.pyx", "nms.c"],
    include_dirs=[numpy.get_include()])],
)

It seems that this is a duplicated question with Attempting to build a cython extension to a python package, not creating shared object (.so) file, but I still want to post it here because there is no much discussions there.

Thank you!

hoefling
  • 59,418
  • 12
  • 147
  • 194
Kai ZHAO
  • 3,088
  • 3
  • 15
  • 18

1 Answers1

6

You don't need the setup script in a subpackage. Just build the extension in the root setup script:

exts = [Extension(name='pytools.nms',
                  sources=["pytools/nms/_nms.pyx", "pytools/nms/nms.c"],
                  include_dirs=[numpy.get_include()])]

setup(
    ...
    packages=['pytools'],
    ext_modules=cythonize(exts)
)

Note that I wrap cythonized extension in cythonize() and use the full module name + full paths to extension sources. Also, since nms is a module in pytools package, including pytools.nms in packages has no effect.

hoefling
  • 59,418
  • 12
  • 147
  • 194
  • Hi hoefling, thanks for your answering! But I have another question after doing as you suggested. The generated .so file is located at `build/lib.linux-x86_64-3.5/pytools/nms.cpython-35m-x86_64-linux-gnu.so`, I cannot import t in `pytools/pytools/nms/__init__.py`. Do you know how to solve it? – Kai ZHAO Apr 26 '18 at 09:29
  • Do you really need the `nms` package if it contains nothing but the `nms` extension? Of course, it's possible to place the extension in the package, so its import becomes `import pytools.nms.nms.nms_func`, but is it really necessary? Remember, [_flat is better than nested_](https://www.python.org/dev/peps/pep-0020/). Or, make `nms` to a separate python distribution (with own setup script), build the `nms` extension there and let `pytools` depend on `nms` package (via `install_requires` keyword in `pytools`' setup script). – hoefling Apr 26 '18 at 12:28
  • Tell me what architecture you envision for `pytools` and I will update the answer accordingly. – hoefling Apr 26 '18 at 12:29
  • For deeper placement of the `nms` extension, it should suffice to add `pytools.nms` to the `packages` list and adapt the extension name to `pytools.nms.nms`. – hoefling Apr 26 '18 at 12:34
  • It seems to be solved since I add 'packages: ['pytools', 'pytools/nms']' into `setup.py`. Thank you anyway. – Kai ZHAO Apr 27 '18 at 14:03
  • Hmm, are you sure it does solve the problem? Because `pytools/nms` is not a valid package name. – hoefling Apr 27 '18 at 15:54
  • @hoefling, you may be able to help with this? https://stackoverflow.com/q/50448026/4080129 – Martino May 21 '18 at 12:44