juliaearth / geoio.jl Goto Github PK

GDAL is a temporary dependency that we rely on to load specific GIS formats. It is very problematic, particularly on Windows, with multiple reports of unsuccessful builds.

Our ultimate goal for the near future is to get rid of this external library, and rely on native Julia backends only.

Currently we do not allow saving a subgrid, but that should be possible if we save the underlying grid with an unview and introduce a MASK column. That way users can save subgrids and reload later with an additional mask step.

It would be useful to write GeoIO.save("foo.csv", geotable) and save the coordinates of the centroids of the geometry column as new columns X, Y, Z, ...

Similarly, we should be able to write GeoIO.load("foo.csv", coords=("X","Y","Z")).

The number of lines is hard to maintain already.

• https://github.com/sintefmath/GeoEnergyIO.jl
• https://github.com/JuliaGeo/ASCIIrasters.jl
• https://github.com/JuliaData/Feather.jl
• https://github.com/apache/arrow-julia

The images are loaded as Julia arrays that are rotated compared to the convention of the X-Y axis. We need to rotate them lazily after loading with ImageIO.jl

• .parquet via GeoParquet.jl
• .png, .jpg, .tif via ImageIO.jl
• .ply via PlyIO.jl
• .gslib via GslibIO.jl

Currently .tif files which contain spatial coordinates with respect to a CRS are being read as an image losing information about he coordinates and also the values do not need to be colors. We can fix it by reading the file with ArchGDAL.jl and make the conversion to Meshes.jl.

Hi,

with Julia 1.10.3 and 1.13.0 I get the following error for opening a .stl file:

using GeoIO
using Meshes

fpath = "/home/user/3DBenchy.stl"
GeoIO.load(fpath)

Base.InvalidCharError{Char}('\xd7')
Stack trace

Here is what happened, the most recent locations are first:

    throw_invalid_char(c::Char) @ [char.jl:86](https://github.com/JuliaLang/julia/tree/0b4590a5507d3f3046e5bafc007cacbbfc9b310b/base/char.jl#L86)
    UInt32 @ char.jl:133
    convert @ char.jl:185
    cconvert @ essentials.jl:543
    lowercase(c::Char) @ [unicode.jl:290](https://github.com/JuliaLang/julia/tree/0b4590a5507d3f3046e5bafc007cacbbfc9b310b/base/strings/unicode.jl#L290)
    map(f::typeof(lowercase), s::String) @ [basic.jl:654](https://github.com/JuliaLang/julia/tree/0b4590a5507d3f3046e5bafc007cacbbfc9b310b/base/strings/basic.jl#L650)
    lowercase @ unicode.jl:623
    _splitline @ stl.jl:160
    (::GeoIO.var"#21#23"{Vector{Tuple{Tuple{Float64, Float64, Float64}, Tuple{Float64, Float64, Float64}, Tuple{Float64, Float64, Float64}}}, Vector{Tuple{Float64, Float64, Float64}}})(io::IOStream) @ stl.jl:37
    open(f::GeoIO.var"#21#23"{Vector{Tuple{Tuple{Float64, Float64, Float64}, Tuple{Float64, Float64, Float64}, Tuple{Float64, Float64, Float64}}}, Vector{Tuple{Float64, Float64, Float64}}}, args::String; kwargs::@Kwargs{}) @ [io.jl:396](https://github.com/JuliaLang/julia/tree/0b4590a5507d3f3046e5bafc007cacbbfc9b310b/base/io.jl#L393)
    open @ io.jl:393
    stlasciiread @ stl.jl:33
    stlraed(fname::String) @ stl.jl:11
    load(fname::String; layer::Int64, fix::Bool, kwargs::@Kwargs{}) @ load.jl:50
    load(fname::String) @ load.jl:21
    [This cell: line 1](http://localhost:1234/edit?id=647d274c-2274-11ef-263e-e155516d9bef#6eb5497e-201a-4905-91df-bceb9f8fa3d4)

    [mesh_object2 = GeoIO.load(fpath)](http://localhost:1234/edit?id=647d274c-2274-11ef-263e-e155516d9bef#6eb5497e-201a-4905-91df-bceb9f8fa3d4)

I wanted to open the common Benchy STL.

Best,

Felix

Hello,

I just realized that the table coming out of the output from NaturalEarth (example https://github.com/JuliaEarth/GeoArtifacts.jl#naturalearth in the Readme) creates a domain whose points are expressed in m, despite having the values which are corresponding to longitude and latitude in degrees.

I understand this as that is the default unit to the new Point type, but the input data from the NaturalEarth repository is in latitude/longitude in degrees so at first glance this might seem misleading.

Is this something intended or should it be fixed or made more explicit in the documentation?
Is it alternatively possible to create the geometries directly using LatLon from CoordRefSystems?
I just realized that you can use LatLon to generate a Point, but this would need some modification in the functions in GeoIO to specify how to handle LatLon when converting from GeoJSON.FeatureCollection to:

The pre-compilation of backends is taking an absurd amount of time. We need to find and fix possible invalidations.

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

The backend packages are Feather.jl for version v1 and possibly Arrow.jl for version v2.

This function should return a table-like object showing:

1. file format extension
2. backend used to load
3. backend used to save

If an entry is not available, it should show as missing.

1 - GeoJSON.jl writes table columns in arbitrary order;
2 - Shapefile.jl saves Chains and Polygons as Multi;
3 - The default number type in GeoJSON.jl is Float32, set this option with numbertype kwarg;
4 - GeoJSON.jl writes Date columns as string because the JSON format has no date types;
5 - KML file is failing to save with Shapefile.jl;

Hello,

I am opening this issue to understand whether there will be willingness (or if there is already an alternative) to move the functionality to translate geometries implementing the interface from GeoInterface into a separate much more lightweight package.

From what I see this is fully enclosed within the conversions.jl file and would only need to depend on Meshes.jl and GeoInterface.jl as a very bare minimum if removing the 4 added methods depending on AbstractGeoTable:

Alternatively and probably more meaningful within the JuliaEarth ecosystem would be to include GeoTables as dependency and maybe also provide the contents of src/utils.jl to allow easily creating a GeoTable as output from this hypotetical new basic package.

My use case is wanting to provide some basic functionality relying on Meshes and GeoTables in one small package of mine https://github.com/disberd/CountriesBorders.jl where I just want to transform the data related to countries coordinates into a Meshes compatible format so that I can exploit all of the nice functionalities of JuliaEarth.

I had already opened a discussion on zulip on the matter here, where I explained that for the package I did not want to depend on GeoIO.jl since this directly depends on all the possible packages dealing with the formats GeoIO supports, which especially due to GDAL (at least in my experience) can cause issues on windows and long precompilation times.

I understand that things are getting better with precompilation but I still rather not have this huge list of indirect dependencies for some vary basic functionality I know will only depend on one specific geojson file.

The discussion in the zulip above led to the suggestion of just generating the table and saving it as binary, but this comes at the risk that the loaded binary format will be very coupled to the specific version of Meshes and GeoTable that generated it.
I think a solution where the file is simply read with GeoJSON.jl (much lighter than GeoIO.jl) and then converted to a GeoTable upon loading is much cleaner.

I have checked a bit the internals here and indeed it would seem to be rather easy to "excise" this functionality into a separate package that GeoIO would depend on.
This would also open a much easier path for other developers in my shoes to adopt the Meshes ecosystem, and if this was to be included inside JuliaEarth it would give more confidence on its support and it being kept up-to-date with changes to GeoTables and Meshes.
I also believe this work of compatibility maintenance would not be higher than what it currently is as this code already lives in GeoIO which is part of the ecosystem and the effort would just be moved from GeoIO to this new light GeoInterfaceConversion package.

Would this be something you would be open to consider?

Sometimes the columns with coordinates have missing values. We should discard all rows with missing coordinates before georefering the table.

Now that the stack supports CRS, we need to account for that in some file formats.

• GeoJSON
• Shapefile
• GeoParquet
• GeoPackage
• GeoTIFF
• NetCDF
• GRIB

MWE:

t = georef(nothing, CartesianGrid(3,3))

GeoIO.save("test.shp", t)

The tests are raising a warning in the test suite, and there are 2 tests failing locally for me.