cannot import module using boost.python

Question

I'm trying to build a simple program using boost.python.
I have the following code:

//greet.cpp
#include <iostream>
#include <boost/python.hpp>

void greet()
{
    std::cout << "hello world!" << std::endl;
}

BOOST_PYTHON_MODULE(greet)
{
    using namespace boost::python;
    def("greet", greet);
}

and the follwing makefile:

PYTHON_VERSION := 2.7

PYTHON_INC := /usr/include/python$(PYTHON_VERSION)
PYTHON_LIB_LOCATION := /usr/lib/python${PYTHON_VERSION}
PYTHON_LIB_FILE := python${PYTHON_VERSION}

BOOST_INC := ~/boost_1_54_0
BOOST_LIB_LOCATION := /home/elyashiv/boost_1_54_0/stage/lib
BOOST_LIB_FILE := boost_python

CC := gcc

CFLAGS := -c -fPIC
CInc := -I ${BOOST_INC} -I ${PYTHON_INC}

CLinkFlags = -shared -Wl,-soname,$@ -L${BOOST_LIB_LOCATION} -l${BOOST_LIB_FILE} -L${PYTHON_LIB_LOCATION} -l${PYTHON_LIB_FILE}

greet.o: greet.cpp

%.so: %.o
    gcc ${CLinkFlags} -o $@ $^

%.o: %.cpp
    ${CC} ${CFLAGS} ${CInc} $^

running make greet.so runs with just a few warnings (redefinition in some boost files).

when I try to import the module in python I get the following:

Python 2.7.3 (default, Apr 10 2013, 05:46:21) 
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import greet
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./greet.so: undefined symbol: _ZNK5boost6python7objects21py_function_impl_base9max_arityEv

what I have done wrong and how to fix it?

edit

the output of ldd greet.so:

linux-gate.so.1 =>  (0x001ee000)
libgcc_s.so.1 => /lib/i386-linux-gnu/libgcc_s.so.1 (0x0055d000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0x0058e000)
/lib/ld-linux.so.2 (0x003a2000)

If you are on Linux, could you also paste the output of `ldd greet.so`? Does that show your boost_python library linked in? — André Anjos, Jul 16 '13 at 11:57
Where is `libboost_python`? It is not getting linked in, therefore, you have the missing symbol. Make sure your compilation is working as expected. — André Anjos, Jul 16 '13 at 14:44
@AndréAnjos as I wrote, I don't get any compilation/linking errors. If some thing is wrong with the linking process, I will like to point it out, because I can't spot it. — elyashiv, Jul 16 '13 at 16:15
I just added a full explanation of your problem below, please upvote if you think it fixes your problem. — André Anjos, Jul 17 '13 at 10:29

score 3 · Accepted Answer · edited May 23 '17 at 12:21

Please note that order is important when you are linking binaries with gcc. The order in which you pass binaries to the linker should be such that the first unit (e.g. your object file) should resolve using the following units (other object files or libraries). In your example, you are linking greet.so wrongly:

%.so: %.o
    gcc ${CLinkFlags} -o $@ $^

This is going to generate a compilation line like this:

gcc -shared -Wl,-soname,greet.so -L/usr/lib -lboost_python -L/usr/lib/python2.7 -lpython2.7 -o greet.so greet.o

Notice that the unit greet.o, which depends on symbols defined in libboost_python.so and libpython2.7.so is coming last and therefore, by the time gcc's linker gets to it, it cannot resolve the undefined symbols anymore. Unfortunately, this is not an error, because the linker cannot know if you want this or not (for example, in Python, libpythonX.Y will be loaded before you can import your code in and it may be skipped therefore - you can dump that library from your command line altogether). So, the default is to ignore all undefined symbols.

You can change that behavior by setting a couple of flags to force undefined symbol detection:

CLinkFlags += -Wl,--unresolved-symbols=report-all

Will report all unresolved symbols as errors and:

CLinkFlags += -Wl,--unresolved-symbols=report-all -Wl,--warn-unresolved-symbols

Will report all unresolved symbols but will still link the binary. You have other options explained on this SO thread. Be warned: this is not what you normally want. Things like libpythonX.Y are never explicitly linked for example, but they are still available at runtime. In practice, you will still get a bunch of fake undefined references that are not worth persuing. The best is to fix your Makefile and make sure your object code comes before the libraries.

To fix your example, just move $^ to the begin of the linking like:

gcc $^ ${CLinkFlags} -o $@

After you compile, and you run ldd, you should see that now libboost_python (and python since you are explicitly linking it) will be linked against greet.so and the loading should work as you expect. I tested it myself locally.

As a rule of thumb, if you have undefined references with gcc and you are confident they should be present on any of the linked code, double-check the order.

Here is a fully working/minimal version of your Makefile that will do the job for your specific case (note we also set the runtime path to your private boost library, so you don't need to set the LD_LIBRARY_PATH as indicated by other answers - see details below):

PYTHON_VERSION := 2.7
PYTHON_INC := /usr/include/python$(PYTHON_VERSION)
BOOST_INC := /home/elyashiv/boost_1_54_0
BOOST_LIB_LOCATION := /home/elyashiv/boost_1_54_0/stage/lib
BOOST_LIB_FILE := boost_python

CC := gcc

CFLAGS := -c -fPIC
CInc := -I${BOOST_INC} -I${PYTHON_INC}

CLinkFlags = -shared -Wl,-soname,$@ -Wl,-rpath,${BOOST_LIB_LOCATION} -L${BOOST_LIB_LOCATION} -l${BOOST_LIB_FILE}

greet.so: greet.o

%.so: %.o
  gcc $^ ${CLinkFlags} -o $@

%.o: %.cpp
  ${CC} ${CFLAGS} ${CInc} $^

By setting the runtime path on the library itself, the runtime linker automatically looks there first before trying the LD_LIBRARY_PATH. If you decide not to do so, then you must set the environment variable LD_LIBRARY_PATH as indicated by other answers.

Your answer is great, but it still not the end of it. merging your answer with doctorloves answer gt rid from all the errors. — elyashiv, Jul 17 '13 at 19:24
Ok, I just added a comment indicating how you can do that using the linker itself. — André Anjos, Jul 17 '13 at 20:35

score 1 · Answer 2 · answered Jul 16 '13 at 08:48

1

Boost python requires the boost python so file. You can add it to your path when you run the python in a variety of ways. I am using

export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH:../ThirdParty/boost_1_52_0/lib/linux64/

answered Jul 16 '13 at 08:48

doctorlove

18,872
2
46
62

It doesn't seem to work. I run `export LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH:~/boost_1_54_0/stage/lib` and got the same error. I'm using the wrong dir? – elyashiv Jul 16 '13 at 11:37
I take it /boost_1_54_0/stage/lib has the boost python so? – doctorlove Jul 16 '13 at 11:46
does that mean I cannot use the extension on a machine that doesn't have the boost libs? if not - how do I do it? – elyashiv Jul 17 '13 at 19:21

score 0 · Answer 3 · answered Jul 17 '13 at 11:29

0

Try these commands. These worked for me.

g++ -c -I/usr/include/python2.7 -fPIC hello.cpp -o hello.o 
g++ -shared -Wl,-soname,"hello.so" -L/usr/local/lib hello.o -lboost_python -fPIC -o hello.so

answered Jul 17 '13 at 11:29

Fr34K

534
6
19

cannot import module using boost.python

3 Answers3

Linked