16

I'm writing some bindings for a C library and am not sure how to configure all this for distribution so it is possible to pip install my package.

Let's say I have the following files:

  • library.c
  • library.h
  • wrapper.py

In order for my wrapper library to work it is necessary to:

  • compile library.c and create a shared library
  • run ctypesgen on library.h to generate the ctypes code

Here are the commands:

  • gcc -Wall -fPIC -c library.c
  • gcc -shared -Wl,-soname,liblibrary.so.1 -o liblibrary.so.1.0 library.o
  • ctypesgen.py library.h -L ./ -l library -o _library.py

Running setup.py will also depend on the user having installed ctypesgen.

I have no idea how to get this all set up so that someone interested in the library can simply pip install library and have all this happen automagically. Anyone able to help?

Eryk Sun
  • 33,190
  • 5
  • 92
  • 111
coleifer
  • 24,887
  • 6
  • 60
  • 75
  • You could put an `import` of the required dependency in a `try/except` at the top of the setup script, and put the code to pip install the dependency in the `except` block – MikeRixWolfe Jun 10 '14 at 17:15
  • No I think you need to put it in `install_requires` or a similar directive. – coleifer Jun 10 '14 at 17:22
  • Yeah, that would be more correct, I'm not sure how either tho. Side note, I recognize your SO username from your blog, I liked your post on markov chaining, used it as a basis for my own markov chaining irc bot :) – MikeRixWolfe Jun 10 '14 at 18:28
  • note that the requirement is a C library - it's "distribution" (library.so + library.h) is not a Python package and would not naturally be listed on PyPI (so dependencies would not work). – amitar Mar 05 '15 at 08:51

1 Answers1

9

The Extensions capability of setuptools/distutils is what you need.

Documentation has more information, in short an example setup.py to do the above would look like the below

from setuptools import setup, find_packages, Extension

extensions = [Extension("my_package.ext_library",
                       ["src/library.c"],
                       depends=["src/library.h"],
                       include_dirs=["src"],
              ),
]
setup(<..>,
     ext_modules=extensions,
)

The .so is generated automatically by setup.py when the module is built. If it needs to link to other libraries can supply a libraries argument list to the extension. See docs(1) for more info.

Since this is built in functionality of setuptools, it works fine with pip and can be distributed (as source code only) on pypi. All source files referenced by the Extension must be present in the distributed pypi archive.

If you want to build distributable binary wheels including native code see manylinux.

danny
  • 5,140
  • 1
  • 19
  • 31
  • 3
    This code is working perfectly for me but the name of the library seems to change depending on the system. For my system, the library is named `ext_library.cpython-36m-x86_64-linux-gnu.so`. How can I predict the name of the package to import it in my python script ? – M-Gregoire Nov 18 '17 at 18:58
  • 2
    You do not need to. The file name is auto-generated depending on interpreter version and architecture. Only the name used when setting up extension is used on import, eg `import my_package.ext_library` from the above. – danny Nov 20 '17 at 10:35
  • To add header files to the tar.gz source package, a MANIFEST.in file is required. `python3 -m build` uses `sdist` to generate the tar.gz and it won’t include header files by default. The `setup.cfg` and `setup.py` files won’t help. – chmike Feb 11 '21 at 13:26