0

I've a python extension module (2.7) (produced by swig), that itself links to a library linked to Python 3.33. Even the module is linked to the version 3.33, when it is instantiated, it calls the PyFunc defined by the 2.7 instead of the ones defined in the 3.33. This happens only on linux , and works good on Windows. Do I need to pass specific linkers flag to avoid this kind of issue ?

Thanks!

Leonardo Bernardini
  • 1,076
  • 13
  • 23
  • Do you've the code of the library which links to Python 3.X ? is this linking done statically or thru dlopen/LoadLibrary ? – Shmil The Cat Dec 10 '14 at 18:43
  • yes I've the code, it's linked dynamically with -lpython on the gcc command line. Do you suggest linking statically ? This will prevent me loading up additional modules with the modules not being up to lookup for the dynamic library.. – Leonardo Bernardini Dec 11 '14 at 12:19

1 Answers1

1

Under Windows, DLLs (Linux shared objects analogue) must resolve all of their external symbols during the link phase, so during the build of your extension, the extension is implicitly linked w/ python 3.0 DLL and everything is working just fine (w/o seeing MSVC command line, I'm almost sure the extension is linked w/ stub library located under something like c:/python33/libs/python33.lib)

Under Linux on the other hand, shared objects default link regime isn't resolving all the external symbols, hence the LD specification of -lpython will probably resolved during runtime to the loaded shared objects which is 2.7 based ...

So you've 2 options :

  1. Since you got the sources, under Linux you can load libpython33.so via dlopendynamically to avoid this "DLL hell"

  2. static approach, you shall specify the exact location of python3.3 i.e. instead of the "vague" -lpython (which resolved to python2.7) something like /usr/lib/python3.3/libpython33.so

Community
  • 1
  • 1
Shmil The Cat
  • 4,548
  • 2
  • 28
  • 37
  • what happens when I open with dlopen ? Does I have to re-fetch all the function pointers from the .so file instead of calling the ones imported from the headers and the -l flag ? – Leonardo Bernardini Dec 11 '14 at 16:02
  • In general, classic usage of dlopen (consuming .so as a plugin) involves the usage of dlsym (fetching function pointers by names etc.) In your case, our sole intent is simply loading the _correct_ .so version (3.3 instead of 2.7) so you don't need use function pointers, your function calls will be resolved to the correct DLL (I of course assuming here, no header/interfaces mismatches which IMO applies to your case) . If I were you, I'd go w/ the 2nd option and see what happens – Shmil The Cat Dec 11 '14 at 16:30
  • can't this mess with the 2.7 invoking the module ? Or the syms are confined to the modules that launches dlopen ? – Leonardo Bernardini Dec 12 '14 at 12:04
  • symbols are not confined to launching module... I'll try clarifying myself better for what (IMO) you should carry out - As said, in my answer, the reason your extension targets 2.7 APIs is b/c it links to the python lib in a "vague" fashion (the -lpython resolved to python2.7) my rec to you is within your makefile (lets neglect the dlopen/dynamic approach) is to specify _explicit_ path to python 3.3 to link against (my 2nd bullet in my answer) - By doing so you will get the following : 1. Py 2.7 is executed 2. Your extension is being loaded/consumed. 3. Py 3.3 loaded and the calls target it – Shmil The Cat Dec 12 '14 at 12:26
  • ok understood it. can't this be verified with ldd ? Before I launch my executables I force the libraries look-up with LD_LIBRARY_PATH, but it's more like the process already has the symbols loaded that matches the ones in the 3.3 lib... by the way I'll try what you suggest.. – Leonardo Bernardini Dec 13 '14 at 19:47