Saturday, November 12, 2011

How to avoid (DY)LD_LIBRARY_PATH with JNI

Some of us are stuck with JNI. We've got a heap of code in C++. Sometimes, we have a heap of code with enough floating point computation in it that the speed advantage of native code is inescapable.

In the simple case, JNI isn't too bad. You make the header, you build the code, and you have a shared library. You can use java.library.path and System.loadLibrary, or you can use System.load and avoid any extra settings when launching java.

Things are not so nice, however, if your C++ code has dependencies on other shared libraries. Listing the directories containing those libraries in java.library.path is not enough.  You'll still get exceptions claiming that your JNI library cannot be loaded. To get rid of those exception, you have to modify PATH, LD_LIBRARY_PATH, or DYLD_LIBRARY_PATH (on Windows, Linux, or OSX respectively). This leads to a world of hurt, particularly when people want to run your code inside a container such as Tomcat.

For Windows, there's a solution involving a Win32 hairball called 'delayed loading'. That's not what this posting will help you with. Perhaps I'll do a writeup some day. At Basis, we worked that out years ago.

Until now, however, we've suffered with LD_LIBRARY_PATH and DYLD_LIBRARY_PATH.

Well, we're not going to suffer any longer. The solution to these problems leaked, finally, into my consciousness, and I've built a testbed to show it off. Here it is on github:

https://github.com/bimargulies/jni-origin-testbed

The code in here shows off the existence of linker options and tools that avoid the need to set those environment variables. On Linux, the critical feature of 'ld' is '-rpath $ORIGIN'. Watch out; it takes some care to actually get the characters '$' 'O' 'R' ... into the ELF file.

On MacOSX, the situation is more complex. MacOSX has this idea that every shared library has a single, proper, installation location, called the 'install path.' Things linked to shared libraries pull that path from the Mach object file, and store it for use at runtime. If you are willing to structure your code as a Framework that follows Apple's conventions for a fixed installation, this all works great.

If not, then there turns out to be a solution. A command, 'install_name_tool', allows you to patch the location where one library (your JNI library) looks for another (its dependencies). The special token '@loader_path' expands to the location of the library itself. Thus, you can express the location of the dependencies by relative path. So long as the JNI libraries live in a fixed location relative to their dependencies, all is well.

1 comment:

Benjamin Douglas said...

Starting with OS X 10.5, there is a way to emulate the $ORIGIN behavior without needing to post-process the binary using install_name_tool. It involves the @rpath setting.

Every lib still has an install_name that contains the path to the file on disk. But now that path can contain a special @rpath token that resolves itself based on the calling library. It takes special coordination between the -install_name setting in the callee and the -rpath setting in the caller.

Here's how you might use this to link an app and a jni so to a main library from different directories.

lib/libmain.dylib: main.o
    g++ ... '-Wl,-install_name,@rpath/libmain.dylib'

bin/app: app.o
    g++ ... '-Wl,-rpath,@executable_path/../lib'

lib/libjni.jnilib: jni.o
    g++ ... '-Wl,-rpath,@loader_path/../lib'

Analytics