Saturday, November 12, 2011

How to avoid (DY)LD_LIBRARY_PATH with JNI

Some of us are stuck with JNI. We've got a heap of code in C++. Sometimes, we have a heap of code with enough floating point computation in it that the speed advantage of native code is inescapable.

In the simple case, JNI isn't too bad. You make the header, you build the code, and you have a shared library. You can use java.library.path and System.loadLibrary, or you can use System.load and avoid any extra settings when launching java.

Things are not so nice, however, if your C++ code has dependencies on other shared libraries. Listing the directories containing those libraries in java.library.path is not enough.  You'll still get exceptions claiming that your JNI library cannot be loaded. To get rid of those exception, you have to modify PATH, LD_LIBRARY_PATH, or DYLD_LIBRARY_PATH (on Windows, Linux, or OSX respectively). This leads to a world of hurt, particularly when people want to run your code inside a container such as Tomcat.

For Windows, there's a solution involving a Win32 hairball called 'delayed loading'. That's not what this posting will help you with. Perhaps I'll do a writeup some day. At Basis, we worked that out years ago.

Until now, however, we've suffered with LD_LIBRARY_PATH and DYLD_LIBRARY_PATH.

Well, we're not going to suffer any longer. The solution to these problems leaked, finally, into my consciousness, and I've built a testbed to show it off. Here it is on github:

The code in here shows off the existence of linker options and tools that avoid the need to set those environment variables. On Linux, the critical feature of 'ld' is '-rpath $ORIGIN'. Watch out; it takes some care to actually get the characters '$' 'O' 'R' ... into the ELF file.

On MacOSX, the situation is more complex. MacOSX has this idea that every shared library has a single, proper, installation location, called the 'install path.' Things linked to shared libraries pull that path from the Mach object file, and store it for use at runtime. If you are willing to structure your code as a Framework that follows Apple's conventions for a fixed installation, this all works great.

If not, then there turns out to be a solution. A command, 'install_name_tool', allows you to patch the location where one library (your JNI library) looks for another (its dependencies). The special token '@loader_path' expands to the location of the library itself. Thus, you can express the location of the dependencies by relative path. So long as the JNI libraries live in a fixed location relative to their dependencies, all is well.

Thursday, July 7, 2011

A maven seder

The other day, Stephen Connolly of the Maven PMC was moved to remark, 'Meh! there's a lot of maven haters out there...' This led me to the following musings, which might be thought of 'A Maven Seder, or, the Four Children of Maven.'

Maven may be one tool, but developers encounter it under very different circumstances. Here's a way to look at those circumstances under four headings, and perhaps reveal something about why developers end up with such wildly different attitudes.

The Developer Who Does Not Know How To Ask

Some developers find Maven by adding themselves to a project that already uses it in some sane and stable fashion. To them, it just works. They type 'mvn'. Their tests run, their jar files appear. Maybe they even follow a recipe to push a release. Likely, if they have this experience, the project they are joining has a Maven-friendly shape: sources in the standard layout, lots of dependencies, no need for complex and exotic scripting. These developers may not fall in love, but they may wonder what all the fuss is about.

The Simple Developer

The simple developer actually sets up a Maven build, but has no particular problem in doing so. His or her inputs are some java sources and some 'in classpath' files. The dependences she or he needs are all sitting out there on Maven Central. The output of the process is a jar file, or, at most, a relatively straightforward release package. He or she copies a simple pom.xml file, makes a few tweaks, and all is well. A copy of Jenkins produces instant automated builds, and a copy of Nexus or Artifactory speeds up the process. No giant stress, no giant strain. So long, of course, as nothing goes wrong. The moment this person has to graduate from 'mvn dependency:tree' to 'mvn -X', they are at risk of becoming ...

The Rebellious Developer

These are the folks who fill blogs and mailing lists with warnings to give Maven a wide berth. How do they get this way? Here are some of the ways:

  • They are given the job of taking a complex build with some other build system and adapting it to Maven. This is a hard job at best. It's made hard and annoying when the Maven-mavens insist on replying to all complaints and questions with variation on, "You should just restructure your entire system to fit in with the Maven way of doing things," instead of "Well, it can be a hard job to adapt a complex, highly-scripted build to Maven. Are you sure you have to? Maybe you just want to learn to publish some artifacts."
  • They have a build that has some incompressible complexity in it. They need to create multiple slight variations for different targets, or complex JNI, or interactions with large datasets. 
  • They run into a problem. Maven works great when it works. When it doesn't work, often the diagnostic process leaves a great deal to be desired. Things seem to happen 'by magic.' If the ordinary log messages aren't informative, the alternative is -X, which spews a gigantic amount of content. At best, this is a needle-in-haystack situation.
The Wise Developer

At the other extreme, we have the developer who has become completely assimilated. He or she has internalized the lifecycle, and so knows exactly what to expect. Little is mysterious or surprising. Chances are, she or he has written a plugin or three, having figured out that writing plugins is often much easier to do (and debug) than convincing the more obscure options on the more obscure plugins to cooperate with each other.

This person is, of course, in danger of turning into one of the voices that ticks off the Rebellious category. Isn't that what family dynamics are like?

Lessons for the Maven Community

In my view, there are some simple lessons here. Maven evangelists should be mindful of what is realistic. Better documentation, error messages, and log message never hurt. 

It's easy to write these sentiments, of course. Acting on them is another story. The Maven ecosystem is now a gigantic stack of code, and the committers for it sometimes seem like short-order cooks in a very busy diner.