Broken WMS due to libjpeg-turbo errors in 2.25.0 about docker-geoserver HOT 10 CLOSED

I think the issue could be either due to the following

• A version of libjturbo that is not compatible with the current geoserver extention
• An upstream issue,
  If you look at the logs you will see the following

This shows that the extension is loaded properly

from docker-geoserver.

@vidlb What CPU architecture are you running on?

@NyakudyaA The error in the trace indicates a linkage error, and is because the plugin is pulling in an older version of libjpeg-turbo than what this Docker image provides.

The Docker image has turbojpeg-wrapper-1.2.1.5.jar – this is quite old. I believe it is downloaded as a transitive dependency from gs-libjpeg-turbo-2.25.0.jar. It does not include any native libraries.

It looks like the error thrown from here: https://github.com/libjpeg-turbo/libjpeg-turbo/blob/49f3485cb58c3c57a513fe0efe00de7cf319708c/java/org/libjpegturbo/turbojpeg/TJCompressor.java#L146

The line numbers don't match with the current upstream main branch, but do match with 1.2.x.

The actual C implementation in JNI is one of the functions whose names start with Java_org_libjpegturbo_turbojpeg_TJCompressor_compress (I don't know which – I forget JNI calling conventions); which is in turbojpeg-jni.c, which is built as libturbojpeg.so. That'll again be based on libjpeg-turbo v1.2.x's JNI interface (which is 13 years old) – but the image is pulling in v3.0.3.

Also, according to the deployment build log, it looks like the first apt install call pulls in libjpeg-turbo8 from Ubuntu as a transitive dependency:

https://github.com/kartoza/docker-geoserver/actions/runs/9140425983/job/25133771692#step:7:285

But then another part of the Dockerfile installs libjpeg-turbo-official from upstream packages:

On aarch64:

• LD_LIBRARY_PATH ends up being /usr/local/tomcat/native-jni-lib:/usr/local/tomcat/native-jni-lib:/usr/lib/jni:/usr/local/apr/lib:/opt/libjpeg-turbo/lib64:/usr/lib:/usr/lib/x86_64-linux-gnu.

  /usr/lib/x86_64-linux-gnu is not correct for non-x86_64 CPUs.

• Upstream's libjpeg-turbo-official SO ends up at /opt/libjpeg-turbo/lib64/libjpeg.so.62.4.0 (C library) and /opt/libjpeg-turbo/lib64/libturbojpeg.so.0.3.0 (JNI wrapper); and includes those Java classes at /opt/libjpeg-turbo/classes/turbojpeg.jar.

• Ubuntu's libjpeg-turbo8 SO ends up at /usr/lib/aarch64-linux-gnu/libjpeg.so.8.2.2 (C library), and doesn't install any Java components. I couldn't find a Java-specific Ubuntu package.

• Both create libjpeg.so symlinks in their respective lib folders, but that's only for C linkage.

So everything is a bit confused; it may only work properly on x86_64 (while my workstation in aarch64, my production deployment is on x86_64).

I think that the solution will be to rebuild gs-libjpeg-turbo linked against the current version of libjpeg-turbo Java libraries.

While you could roll back to libjpeg-turbo 1.2.x to match what that plugin uses, it likely has security problems and is end of life and not supported by upstream.

from docker-geoserver.

@vidlb What CPU architecture are you running on?

My CPU is x86_64

from docker-geoserver.

My CPU is x86_64

Hmm, OK. My theories about library paths (while an issue) are probably not the cause.

I've just had a poke at my production instance (running a fork of this image) with GeoServer 2.25.1 on x86_64 using a vector layer (in PostGIS) over WMS.

Outputting as PNG is working fine there, but outputting as JPEG is returning the same error you're seeing.

I've also tried this out with on aarch64 and an ESRI ShapeFile (vector) layer, and seeing the same again.

The error you're seeing with PNG is while loading a raster JPEG layer, and I don't have any raster layers loaded there to try out.

I've done some more digging into the dependency chain, and it looks like GeoSolutions IT's fork of the libjpegturbo Java libraries was made 11+ years ago, and I can't find any C code in that repository for what the JNI libraries are linking to... so they'll still be trying to hit interfaces that have broken ABI compatibility.

I've filed the issue upstream (with more detail): geosolutions-it/imageio-ext#305

As for what we could do here, I found removing the libjpeg-turbo GeoServer plugin from the list of required plugins fixed it in my local fork.

My fork has #659 already applied; so I just removed the libjpeg-turbo-plugin line from build_data/required_plugins.txt, and added it to build_data/stable_plugins.txt.

from docker-geoserver.

Can you try the 2.25.1 image, we now can disable the plugin if not needed by using the env ACTIVE_EXTENSIONS=

from docker-geoserver.

I just tried after adding the empty ACTIVE_EXTENSIONS= env but when I remove the JVM opt the plugin is loaded as usual, am I supposed to start with an empty DATA_DIR ? Is there a way to avoid this ?

from docker-geoserver.

@vidlb you will need to adjust it to become

ACTIVE_EXTENSIONS=control-flow-plugin,csw-iso-plugin,csw-plugin,gdal-plugin,inspire-plugin,monitor-plugin,pyramid-plugin,vectortiles-plugin,wps-plugin

You will need to adjust the env to skip the default plugin

from docker-geoserver.

All right it seems to work !
Regarding the ACTIVE_EXTENSIONS env, people might expect to see every plugin disabled when empty, don't you think ?
Empty could be handled differently than unset.

from docker-geoserver.

If ACTIVE_EXTENSIONS is empty it should install all default plugins, this is for backward compatibility. We just need to make it explicit that this variable is meant to give users the power to select which default plugins should not be activated. Any wording suggestions

from docker-geoserver.

I find it hard to explain for the empty case, would be simpler if the variable was DISABLED_EXTENSIONS, but may be something like

Use this variable to specify a set of plugins to enable, if left empty or unset the following will be enabled : [list of default plugins]

Edit: libjpeg-turbo should be discouraged

from docker-geoserver.