5

I want to distribute a Python package which has a closed source dependency. I am using setup.py and everything works if I also do the compilation with setup.py.

Neither answers to this question nor answers to that question solve my problem.

I have the following file structure:

.
├── closed_source
│   ├── compiled.so
├── python_modules
│   ├── file1.py
│   ├── file2.py
│   ├── ...
│   └── __init__.py
└── setup.py

I also tried to include compiled.so in python_modules. In file1.py I use import compiled which fails.

The following works, but silently fails to include the dynamic library:

setup(
    name='my_package',
    version=0.1,
    packages=['python_modules'],
    package_dir={'python_modules': 'python_modules'},
    package_data={'': ['closed_source/compiled.so']}, # also tried using key compiled
    include_package_data=True,
)
Niki
  • 738
  • 8
  • 17

1 Answers1

3

You will have to vendor the dependency. The easiest way to do that is to include the dependency inside your python package, not outside.

setuptools requires you to include a MANIFEST.in file to include non-package, non-python files in your distribution.

See this project for an example of how to do that.

Your project structure should look something like this:

my_package
|-- vendor
    |-- compiled.so
|-- __init__.py
|-- file1.py
|-- file2.py
setup.py

You would also need to import your vendored library using the additional relative prefix

from .vendor import compiled
Brendan Abel
  • 35,343
  • 14
  • 88
  • 118
  • Perfect, worked. It actually works without a `Manifest.in` file. – Niki Jun 14 '21 at 01:06
  • 1
    @Niki It's possible `setuptools` recognizes your file extension as a package file. For resource files like txt, jpg, json, etc., it normally won't pick them up by default. – Brendan Abel Jun 14 '21 at 01:08