Version 0.8.3 broken...? about lmdbjava HOT 20 CLOSED

Try comparing the output of mvn dependency:tree on the project for 0.8.2 vs 0.8.3. Take a close look at any com.github.jnr dependencies. LmdbJava 0.8.3 updated the version and perhaps you've got an older version directly declared in your POM or being brought in transiently by another library. If that doesn't work, can you reproduce the issue on a different machine (eg a developer's workstation)? Which operating system is being used?

from lmdbjava.

Try comparing the output of mvn dependency:tree on the project for 0.8.2 vs 0.8.3. Take a close look at any com.github.jnr dependencies. LmdbJava 0.8.3 updated the version and perhaps you've got an older version directly declared in your POM or being brought in transiently by another library.

I suspect the issue has to be with the binding itself, because the same tests run fine on Windows, but they're crashing as reported on RedHat 8.
I'll anyway give it a try and let you know, just to be double sure.

from lmdbjava.

Below, the dependency trees for lmdbjava:

[INFO] +- org.lmdbjava:lmdbjava:jar:0.8.2:compile
[INFO] | +- com.github.jnr:jnr-constants:jar:0.9.15:compile
[INFO] | - com.github.jnr:jnr-ffi:jar:2.2.2:compile
[INFO] | +- com.github.jnr:jffi:jar:1.3.1:compile
[INFO] | +- com.github.jnr:jffi:jar:native:1.3.1:runtime
[INFO] | +- org.ow2.asm:asm:jar:9.1:compile
[INFO] | +- org.ow2.asm:asm-commons:jar:9.1:compile
[INFO] | +- org.ow2.asm:asm-analysis:jar:9.1:compile
[INFO] | +- org.ow2.asm:asm-tree:jar:9.1:compile
[INFO] | +- org.ow2.asm:asm-util:jar:9.1:compile
[INFO] | +- com.github.jnr:jnr-a64asm:jar:1.0.0:compile
[INFO] | - com.github.jnr:jnr-x86asm:jar:1.0.2:compile

[INFO] +- org.lmdbjava:lmdbjava:jar:0.8.3:compile
[INFO] | +- com.github.jnr:jnr-constants:jar:0.10.4:compile
[INFO] | - com.github.jnr:jnr-ffi:jar:2.2.13:compile
[INFO] | +- com.github.jnr:jffi:jar:1.3.10:compile
[INFO] | +- com.github.jnr:jffi:jar:native:1.3.10:runtime
[INFO] | +- org.ow2.asm:asm:jar:9.2:compile
[INFO] | +- org.ow2.asm:asm-commons:jar:9.2:compile
[INFO] | +- org.ow2.asm:asm-analysis:jar:9.2:compile
[INFO] | +- org.ow2.asm:asm-tree:jar:9.2:compile
[INFO] | +- org.ow2.asm:asm-util:jar:9.2:compile
[INFO] | +- com.github.jnr:jnr-a64asm:jar:1.0.0:compile
[INFO] | - com.github.jnr:jnr-x86asm:jar:1.0.2:compile

I can see updates in the dependencies and they match the versions in your pom, so it should be fine.

from lmdbjava.

In the stacktraces, the only references to lmdbjava are

java.lang.NoClassDefFoundError: Could not initialize class org.lmdbjava.Library
at org.lmdbjava.Env$Builder.open(Env.java:521)
at org.lmdbjava.Env$Builder.open(Env.java:547)

from lmdbjava.

I suspect the issue has to be with the binding itself, because the same tests run fine on Windows, but they're crashing as reported on RedHat 8.

Apart from the JNR library update, there are no changes to 0.8.3 at all which would manifest themselves as early as the Library static initializer.

We also CI test on both Windows and Linux. Indeed I only use Linux (and on dozens of servers) so I'm surprised you're seeing this. Have you tried it on another Linux machine by any chance? Also could you please try the following on your failing Linux box:

git clone https://github.com/lmdbjava/lmdbjava.git
cd lmdbjava
mvn test

In the stacktraces, the only references to lmdbjava are

Is there a "Caused by:" clause further down the stack trace?

from lmdbjava.

I suspect the issue has to be with the binding itself, because the same tests run fine on Windows, but they're crashing as reported on RedHat 8.

Apart from the JNR library update, there are no changes to 0.8.3 at all which would manifest themselves as early as the Library static initializer.

We also CI test on both Windows and Linux. Indeed I only use Linux (and on dozens of servers) so I'm surprised you're seeing this. Have you tried it on another Linux machine by any chance? Also could you please try the following on your failing Linux box:

git clone https://github.com/lmdbjava/lmdbjava.git
cd lmdbjava
mvn test

I'll try.

In the stacktraces, the only references to lmdbjava are

Is there a "Caused by:" clause further down the stack trace?

Not really, unfortunately.

from lmdbjava.

Output of mvn test: lmdbjava.txt

More info about the env:

mvn -version
Apache Maven 3.8.5 (3599d3414f046de2324203b78ddcf9b5e4388aa0)
Maven home: /home/tk35c/Applications/apache-maven-3.8.5
Java version: 11.0.11, vendor: Azul Systems, Inc., runtime: /usr/lib/jvm/zulu-11
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-1160.62.1.el7.x86_64", arch: "amd64", family: "unix"

/usr/lib/jvm/zulu-11/bin/java -version
openjdk version "11.0.11" 2021-04-20 LTS
OpenJDK Runtime Environment Zulu11.48+21-CA (build 11.0.11+9-LTS)
OpenJDK 64-Bit Server VM Zulu11.48+21-CA (build 11.0.11+9-LTS, mixed mode)

from lmdbjava.

The mvn test output narrows down the problem:

Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type `POINTER`, original error message follows: java.lang.UnsatisfiedLinkError: Unable to execute or load jffi binary stub from `/tmp`. Set `TMPDIR` or Java property `java.io.tmpdir` to a read/write path that is not mounted "noexec".
/home/tk35c/lmdbjava/jffi6223208339754133543.so: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by /home/tk35c/lmdbjava/jffi6223208339754133543.so)

Is there a writable /tmp directory, and is /lib64/libc.so.6 available?

from lmdbjava.

Hi @benalexau.
The /tmp folder is available as well as libc.so.6.

ls -l /lib64/libc.so.6
lrwxrwxrwx 1 root root 12 May 25 2022 /lib64/libc.so.6 -> libc-2.17.so

To my understanding of the error above, it seems more the case that GLIBC_2.27 is missing and therefore it would be nice to know what are the underlying OS requirements when running LMDB.

from lmdbjava.

Are you able to install a later version of glibc? The latest version is 2.37.

from lmdbjava.

I tried to update it, but with no luck.
Seems like there is no newer version supported for RHEL 7.x

from lmdbjava.

I don't use RHEL so I can't offer more precise instructions, but one simple workaround may be to install LMDB from an official RPM package and then set the lmdbjava.native.lib system property when launching the JVM.

from lmdbjava.

Ah, that's nice.
There would be no impact (also in the future) between the lmdbjava version and the one on the OS?

EDIT: Would it otherwise possible to backport #185 to 0.8.2? That's the reason why I'm updating actually :)

from lmdbjava.

There would be no impact (also in the future) between the lmdbjava version and the one on the OS?

That's correct. If you installed from an OS package and always launched with the system property set to that location it would always ignore the LMDB bundled inside the LmdbJava JAR.

Would it otherwise possible to backport #185 to 0.8.2? That's the reason why I'm updating actually

It is possible but on the balance of probability you're likely to eventually come across some bug or dependency or other requirement that necessitates an upgrade and you're back to where you are at the moment. The system property approach at least gives you a sustainable solution pending RedHat updating glibc.

Another approach that may be worth exploring is to just make a local Maven project which depends on LmdbJava 0.8.3 but then adds in the earlier LmdbJava Native dependency. Then you have a fully self-contained Java-only approach, and don't need to do anything at all with RedHat packages or JVM system properties.

from lmdbjava.

I tried using the lmdbjava.native.lib as system property while running the jar with a value of /usr/openv/pdde/vpfs/lib/liblmdb.so and with /usr/lib64/liblmdb.so.0.0.0.0, but then lmdbjava it's still extracting something and, this time, not in the /tmp folder, but in the working directory, and also still failing:

Caused by: java.lang.UnsatisfiedLinkError: could not get native definition for type POINTER, original error message follows: java.lang.UnsatisfiedLinkError: Unable to execute or load jffi binary stub from /tmp. Set TMPDIR or Java property java.io.tmpdir to a read/write path that is not mounted "noexec".
/export/home/eodm/eod-master/jffi12499849779839545993.so: /lib64/libc.so.6: version `GLIBC_2.27' not found (required by /export/home/eodm/eod-master/jffi12499849779839545993.so)

Another approach that may be worth exploring is to just make a local Maven project which depends on LmdbJava 0.8.3 but then adds in the earlier LmdbJava Native dependency. Then you have a fully self-contained Java-only approach, and don't need to do anything at all with RedHat packages or JVM system properties.

How could this possibly work? For what I've seen, lmdbjava is a shaded fat jar which already has the libs in it. Even if I depend on the native libs before it, wouldn't it just use the libs it has already in itself?

from lmdbjava.

How could this possibly work? For what I've seen, lmdbjava is a shaded fat jar which already has the libs in it. Even if I depend on the native libs before it, wouldn't it just use the libs it has already in itself?

If you take a look in Library you will see it detects if lmdbjava.native.lib is non-null and then will skip extracting the bundled LMDB native library. Instead it will use the path provided. I suggest there's an issue with how the system property was passed to the JVM. I suggest you try accessing that system property (and perhaps opening a File and displaying its size or similar to guarantee it really exists and is accessible) from whatever code runs before you attempt to use LmdbJava.

from lmdbjava.

@benalexau my "how could this possibly work" was referring to

Another approach that may be worth exploring is to just make a local Maven project which depends on LmdbJava 0.8.3 but then adds in the earlier LmdbJava Native dependency. Then you have a fully self-contained Java-only approach, and don't need to do anything at all with RedHat packages or JVM system properties.

Regarding the system property, indeed it gets read and used, but then I was getting the other error I've listed above about ffi.

from lmdbjava.

then I was getting the other error I've listed above about ffi.

Traced this back to issue jnr/jffi#138. This appears to be corrected in JFFI 1.3.11 which was released 3 days ago.

Would you be able to modify your project to depend on that version and report back if it works? If so I'll update the versions in LmdbJava.

from lmdbjava.

then I was getting the other error I've listed above about ffi.

Traced this back to issue jnr/jffi#138. This appears to be corrected in JFFI 1.3.11 which was released 3 days ago.

Would you be able to modify your project to depend on that version and report back if it works? If so I'll update the versions in LmdbJava.

I can give it a try.

from lmdbjava.

In the end we're migrating to RHEL 8 which supports glibc 2.27 and therefore it's for us not worthy anymore to invest time in trying to make the older version work.
Thank you for your help anyway :)

from lmdbjava.