How to change where a shared library is looked for?

Question

I have a dynamic library libOne.dylib written in Rust and I'd like to call it from Java over JNI. I have the sources for the Rust part but can't change them so I've written a thin C wrapper as the JNI counterpart. I can compile and link my wrapper as libTwo.dylib and load it into the JVM.

Unfortunately, the next thing the JVM is doing is loading libOne by looking into the folder where this library was placed during the time it was built so effectively I can't even start my Java application in any other environment without creating the same path where the Rust project is located on my machine and placing the libOne inside it.

Is it possible to change this behavior so that the library looked for in the same folder as the wrapper? I would prefer to completely embed the libOne into libTwo, but I'm not sure if it is possible/feasible. Ideally, I'd like to package the resulting library(ies) directly into the fat jar.

This is how I'm creating the wrapper library.

LIBS="$(pwd)/lib"

gcc \
  -dynamiclib \
  -shared \
  -I$JAVA_HOME/include \
  -I$JAVA_HOME/include/darwin \
  -I$JAVA_HOME/include/linux \
  *.c \
  -o $LIBS/libTwo.dylib \
  -L$LIBS \
  -lOne \
  -Wl,-rpath,.

It is placed directly in the Java project's lib folder but still looks up in the path it was initially built.

The exception looks like following:

java.lang.UnsatisfiedLinkError: /Users/user/LIB-TWO-PROJECT/lib/lobTwo.dylib: 
dlopen(/Users/user/LIB-TWO-PROJECT/lib/libTwo.dylib, 1): 
Library not loaded: /Users/user/LIB-ONE-PROJECT/target/release/deps/libOne.dylib
  Referenced from: /Users/user/LIB-TWO-PROJECT/lib/lobTwo.dylib
  Reason: image not found
    at java.lang.ClassLoader$NativeLibrary.load(Native Method)
    at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
    at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
    at java.lang.Runtime.load0(Runtime.java:809)
    at java.lang.System.load(System.java:1086)

Please note that the path for libOne is the LIB-ONE-PROJECT release folder (target/release/deps) and not the LIB-TWO-PROJECT/lib where both libs are placed.

See this answer https://stackoverflow.com/a/53155508/2543253 That's for Swift, but the same is true for any other dylib. Search the *net for rpath — user2543253, Nov 03 '20 at 15:52
Sorry that this is just a comment and not an answer, I'd gladly accept it. This was exactly what I was looking for. In fact, this is just a one liner: `install_name_tool -change /Users/user/LIB-ONE-PROJECT/target/release/deps/libOne.dylib @loader_path/libOne.dylib /Users/user/LIB-TWO-PROJECT/lib/lobTwo.dylib` in the case if both libraries are colocated in the same folder. — Slava Schmidt, Nov 10 '20 at 15:46

score 1 · Answer 1 · answered Nov 02 '20 at 09:01

The question is: is it possible to change this behavior so that the library looked for in the same folder as the wrapper?

It's not entirely clear what the issue is as you provide no repro case (and I don't have a Java environment on this machine anyway) but I expect the problem is that Java looks for the "native library" on its search path, but the native library then looks the sub-library on the "system" dll path(s).

So you would need to add the Java search path (at least the one containing native libraries) to the system's load path e.g. PATH on windows (same as binaries), LD_LIBRARY_PATH on linux (possibly BSDs as well? not sure), and DYLD_FALLBACK_LIBRARY_PATH (or DYLD_LIBRARY_PATH but that's riskier) on OSX.

I would prefer to completely embed the libOne into the libTwo but not sure if it is possible/feasible.

IIRC can compile the Rust library as a static library (a .a file), the (statically) link that into the final dylib. I've never needed to do this so I can't really help any further, but how to link static library into dynamic library in gcc seems like a good start for resolving issues.

`but the native library then looks the sub-library on the "system" dll path(s). ` - this it the problem, the system looks at the compile-time path of the first library and completely ignores all settings of the system it is running at — Slava Schmidt, Nov 02 '20 at 11:25

Aska · Answer 2 · 2020-11-02T13:15:31.713

0

You may execute your Java application defining the -Djava.library.path VM argument.

This way, you will instruct the JVM where to search for native libs.

As stated in the Javadocs:

java.library.path: List of paths to search when loading libraries

So you can define all the places that contain the native libs that you need. Eg:

-Djava.library.path=/opt/path1:/opt/path2

For linux, use : for separator. For windows, use ;

edited Nov 02 '20 at 13:15

answered Nov 02 '20 at 12:46

Aska

63
1
7

I do define this parameter and my wrapper library is found but the rust library placed in the same folder is not. Instead, I'm getting an UnsatisfiedLinkError. I've added the exception to the question. – Slava Schmidt Nov 02 '20 at 15:36

How to change where a shared library is looked for?

2 Answers2