Comments (7)
rouault
commented on August 16, 2024
For what is worth Java tests pass fine on the "build-mac" job of .github/workflows/cmake_builds.yml . For example https://github.com/OSGeo/gdal/actions/runs/9678601962/job/26702745129 has:
test 1
Start 1: java_GDALOverviews
1: Test command: /Users/runner/hostedtoolcache/Java_Temurin-Hotspot_jdk/21.0.3-9.0/arm64/Contents/Home/bin/java "-Djava.library.path=/Users/runner/work/gdal/gdal/build/swig/java" "-cp" "gdal.jar:build/apps" "GDALOverviews" "tmp_test/byte.tif" "NEAREST" "2" "4"
1: Working Directory: /Users/runner/work/gdal/gdal/build/swig/java
1: Environment variables:
1: PATH=/Users/runner/work/gdal/gdal/build/apps:/Users/runner/work/gdal/gdal/build/swig/python/gdal-utils/scripts:/opt/homebrew/lib/ruby/gems/3.0.0/bin:/opt/homebrew/opt/[email protected]/bin:/Users/runner/.local/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/Users/runner/.cargo/bin:/usr/local/opt/curl/bin:/usr/local/bin:/usr/local/sbin:/Users/runner/bin:/Users/runner/.yarn/bin:/Users/runner/Library/Android/sdk/tools:/Users/runner/Library/Android/sdk/platform-tools:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/usr/bin:/bin:/usr/sbin:/sbin:/Users/runner/.dotnet/tools
1: LD_LIBRARY_PATH=/Users/runner/work/gdal/gdal/build:
1: PYTHONPATH=/Users/runner/work/gdal/gdal/build/swig/python/
1: GDAL_DRIVER_PATH=dummy
1: GDAL_DATA=/Users/runner/work/gdal/gdal/build/data
1: Test timeout computed to be: 1500
1: Raster dataset parameters:
1: Projection: PROJCS["NAD27 / UTM zone 11N",GEOGCS["NAD27",DATUM["North_American_Datum_1927",SPHEROID["Clarke 1866",6378206.4,294.978698213898,AUTHORITY["EPSG","7008"]],AUTHORITY["EPSG","6267"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4267"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",-117],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","26711"]]
1: RasterCount: 1
1: RasterSize (20,20)
1: 0...10...20...30...40...50...60...70...80Band 1 :
1: DataType: 1
1: Size (20,20)
1: PaletteInterp: Gray
1: OverView 0 :
1: DataType: 1
1: Size (10,10)
1: PaletteInterp: Gray
1: OverView 1 :
1: DataType: 1
1: Size (5,5)
1: PaletteInterp: Gray
1: Completed.
1: Use: gdalread tmp_test/byte.tif outfile.png [overview] to extract a particular overview!
1: ...90...100 - done.
5/44 Test #1: java_GDALOverviews .................... Passed 0.22 sec
One difference being that this CI config is on the build artifacts directly, not on the result of "make install", and CMake does some magic regarding changing rpath when installing. What happens if you try to use libgdalalljni.dylib under $build_dir/swig/java ?
I can't really help more, not having direct access to a Mac
from gdal.
tbarsballe
commented on August 16, 2024
I just tried that (locally at least, still need to try on the runner), with no luck - same failure.
The Java version may be something else to pursue - 11 to 21 is a fairly big jump.
I'll do some more tests and report back...
from gdal.
rouault
commented on August 16, 2024
I just tried that (locally at least, still need to try on the runner), with no luck - same failure.
so "ctest -R java" doesn't work on your local build?
The Java version may be something else to pursue - 11 to 21 is a fairly big jump.
We haven't done any specific change to handle java versions. No idea what has changed. My Java knowledge is stuck somewhere at around Java 1.6 :-)
from gdal.
tbarsballe
commented on August 16, 2024
I just tried that (locally at least, still need to try on the runner), with no luck - same failure.
so "ctest -R java" doesn't work on your local build?
I just tried the Java gdalinfo invocation using local paths (build/swig/java) instead.
"ctest -R java" does indeed work locally, which is promising.
And pulling out one of the specific commands works fine too
"/Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home/bin/java" "-Djava.library.path=/Users/torben.barsballe/Documents/data/programs/gdal-3.7.0/build/swig/java" "-cp" "gdal.jar:build/apps" "GDALOverviews" "tmp_test/byte.tif" "NEAREST" "2" "4"
However, dropping the path from in front of java resulted in a failing state, despite both binaries reporting the same version information
java "-Djava.library.path=/Users/torben.barsballe/Documents/data/programs/gdal-3.7.0/build/swig/java" "-cp" "gdal.jar:build/apps" "GDALOverviews" "tmp_test/byte.tif" "NEAREST" "2" "4"
Conversely, including the path in the gdalinfo call got that one working
/Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home/bin/java -Djava.library.path=$DYLD_LIBRARY_PATH -classpath $PWD/swig/java/gdal.jar:$PWD/swig/java/build/apps gdalinfo
Weird. Seems like this issue might actually be related to how the MacOS java distribution is packaged; it produces both the /Library/Java/JavaVirtualMachines directory, and a seperate /usr/local/bin/java binary which is not a symlink.
Dropping /Library/Java/JavaVirtualMachines/temurin-11.jdk/Contents/Home/bin in front of /usr/local/bin on my PATH also seems to work.
The Java version may be something else to pursue - 11 to 21 is a fairly big jump.
We haven't done any specific change to handle java versions. No idea what has changed. My Java knowledge is stuck somewhere at around Java 1.6 :-)
There's certainly been a lot of changes, many of which involve package visibility and such. Though I'm mostly wondering if there's some strange Mac-specific JNI issues hiding somewhere.
from gdal.
tbarsballe
commented on August 16, 2024
Interestingly, ctest, along with the various path tinkering that worked for me locally, failed on the runner - it appears all the linking failures in that environment came down to an architecture issue, which was hidden by the poor error-reporting of the Java 11 JNI.
Switching to Java 21 gives much more detail - the important part being '/Users/runner/work/geotools/geotools/gdal-3.9.0/build/swig/java/libgdalalljni.dylib' (mach-o file, but is an incompatible architecture):
java.lang.UnsatisfiedLinkError: /Users/runner/work/geotools/geotools/gdal-3.9.0/build/swig/java/libgdalalljni.dylib: dlopen(/Users/runner/work/geotools/geotools/gdal-3.9.0/build/swig/java/libgdalalljni.dylib, 0x0001): tried: '/Users/runner/work/gdal/lib/libgdalalljni.dylib' (no such file), '/Users/runner/work/gdal/lib/jni/libgdalalljni.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/Users/runner/work/gdal/share/java/libgdalalljni.dylib' (no such file), '/opt/homebrew/lib/libgdalalljni.dylib' (no such file), '/usr/lib/libgdalalljni.dylib' (no such file, not in dyld cache), '/Users/runner/work/geotools/geotools/gdal-3.9.0/build/swig/java/libgdalalljni.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/geotools/geotools/gdal-3.9.0/build/swig/java/libgdalalljni.dylib' (no such file), '/Users/runner/work/geotools/geotools/gdal-3.9.0/build/swig/java/libgdalalljni.dylib' (mach-o file, but is an incompatible arch
Error: Exception in thread "main" java.lang.UnsatisfiedLinkError: 'long org.gdal.ogr.ogrJNI.CreateCodedFieldDomain(java.lang.String, java.lang.String, int, int, java.util.HashMap)'
at org.gdal.ogr.ogrJNI.CreateCodedFieldDomain(Native Method)
at org.gdal.ogr.ogr.CreateCodedFieldDomain(ogr.java:135)
at test_ogrfielddomains.testOGRCodedValueTypemap(test_ogrfielddomains.java:42)
at test_ogrfielddomains.main(test_ogrfielddomains.java:57)
It appears setup-java was pulling down an x64 JDK; forcing to use arm64 makes everything work (despite the fact that the default arch is supposed to be autodetected by that action). Rolling back to Java 11, and everything's (more or less) still working
from gdal.
rouault
commented on August 16, 2024
Rolling back to Java 11, and everything's (more or less) still working
excellent, I guess this ticket can be closed then?
from gdal.
tbarsballe
commented on August 16, 2024
Rolling back to Java 11, and everything's (more or less) still working
excellent, I guess this ticket can be closed then?
Yes, ultimately it turned out that all the issues were lower-level than GDAL - architecture mismatches, poor error reporting by JNI, etc.
The root causes were all so opaque that I feel like some documentation improvements might be merited, but at the same time I'm not sure there's anything generalizable enough I can pull out of all this to write such documentation.
from gdal.
Related Issues (20)
- Contrary to man page gdalbuildvrt does not fetch from the last file when there is spatial overlapping HOT 3
- ogrinfo: input file should be optional HOT 4
- MySQL-Support in gdal on Mac missing HOT 1
- Support for GML <OrientableCurve> tag HOT 4
- OSM Toner doesn't work in gdal2tiles HOT 1
- Add dataset creation options for GML driver to place geometry field after attribute fields HOT 7
- Invalid nodata value in gdal_rasterize when output is int64 HOT 3
- Converting non-LZW-compressed GeoTIFF to PNG produces black image HOT 1
- Tiff color table component values seem wrong (minus one from what they should be) HOT 3
- ogr2ogr does not see Postgresql schemas on gdal 3.9.0 HOT 10
- Unable to write KMZ to Cloud Storage
- Issue in gdal.Grid called from python bindings HOT 1
- memory leak gdalenhance.cpp
- GMLAS driver: not correct geometry fields HOT 4
- Task: Remove CPLList
- gml writer: FeatureCollection's envelope SRS (3.9.1 regression) HOT 2
- gdal_retile moves tiles a few KM from where their source GeoTIFF's footprint was HOT 5
- Calculated clipsrc area is too small for pdf to geojson ogr2ogr conversion HOT 3
- gdallocationinfo: extra line is output if coordinate is outside raster extent (3.9.0 regression)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gdal.