move GeoInterface conversions to a separate lightweight package about geoio.jl HOT 5 CLOSED

Thanks for confirming I could reuse the code as proposed.

Regarding the filtering functionality I will try to see if it can be added once we stabilize the API a bit more.

We also have a core internal package that implemented a lot of things that are now in CoordRefSystem for dealing with visibility between satellites and points on ground.
I have been thinking about checking how difficult or impactful on performance using CoordRefSystem internally would be so I might come again to open issues or ask questions in other packages of JuliaEarth.

Overall I think you are doing a very commendable effort into trying to harmonize these fields in Julia and I would like to re-use and contribute back to the ecosystem when possible :)

from geoio.jl.

Hi @disberd , thanks for raising the issue here. Appreciate your attempts to improve the ecosystem.

My use case is wanting to provide some basic functionality relying on Meshes and GeoTables in one small package of mine https://github.com/disberd/CountriesBorders.jl where I just want to transform the data related to countries coordinates into a Meshes compatible format so that I can exploit all of the nice functionalities of JuliaEarth.

Can you please confirm that you are creating CountriesBorders.jl because you don't want to take GeoArtifacts.jl as a dependency (which depends on GeoIO.jl)? As explained in Zulip, the Natural Earth dataset is already available as a GeoTable with Meshes.jl geometries that are compatible with our ecosystem. Can you please explain how CountriesBorders.jl differs from GeoArtifacts.NaturalEarth?

I think a solution where the file is simply read with GeoJSON.jl (much lighter than GeoIO.jl) and then converted to a GeoTable upon loading is much cleaner.

It seems to me that your main concern with the current situation is the heavy GeoIO.jl dependency. I would like to ask you again: are you concerned with the heavy dependency in CountriesBorders.jl or in some other user-facing app that you are building that depends on CountriesBorders.jl? In the former case, I'd argue that CountriesBorders.jl is not needed, especially if you simply use/contribute to GeoArtifacts.jl (please let me know if there are feature differences other than just loading the boundaries). In the latter case, your user-facing app will be compiled anyways, and you will be able to load many other formats in the future, without manual intervention. Issue #23 is something that we need to solve in parallel. Contributions are welcome.

I did not want to depend on GeoIO.jl since this directly depends on all the possible packages dealing with the formats GeoIO supports, which especially due to GDAL (at least in my experience) can cause issues on windows and long precompilation times.

We do share your second concern very much. We've been working really hard over the years to get rid of this GDAL madness that is spread across various programming languages. Our ultimate goal is to completely remove this dependency in GeoIO.jl. However, this removal depends on the existence of more native Julia packages to load file formats that are not supported yet. We should probably make a list of the missing formats to make this road map more explicit.

In summary, and answering to your request: we do not envision a promising direction with a separate package for geometry conversions.

from geoio.jl.

Thanks for the quick reply.

Indeed my problem is that CountriesBorders is a dependency (direct or indirect) of a number of private packages we develop inernally for work.
I decided to make this package public as it's simple, does not rely on any internal data and might be useful to someone else.

The problem with the huge GeoIO dependency is that precompilation does not really help in CI where everything is recompiled again from scratch (we use GitLab at work with a self hosted instance so I do not have access to all the julia actions on github that might help with caching).
We also test both on windows and linux so for the foreseeable future I do not want to have a dependency on GeoIO within this package due to the massive increase in CI time and significant issues with windows CI.

I am working a new PR to allow a more recent version of Meshes here:
disberd/CountriesBorders.jl#11

For the time being, considering your reply about not being willing to move this outside of GeoIO, it seems like I can just live with adding some internal compat based on the files from GeoIO.jl as done here:
https://github.com/disberd/CountriesBorders.jl/pull/11/files#diff-900f3fdd76930cfba1347564aa40447c497c942e2138e935e99ca9c9d9f7b4e4

I added the full copyright from this repo in the comment at the beginning of the file, let me know if this is sufficient or if I have to handle the copyright differently.

EDIT: @juliohm Regarding how this differs from what is available in GeoArtifacts.NaturalEarth, not much in the raw content but I do export functionality in CountriesBorders to allow easy selection of specific countries and downfiltering based on region/name.
We then use this downselection to filter out locations of users on ground depending on whether they are on downselected country/region or not.
It also provides some basic plotting functionality based on Plotly which we use internally.

from geoio.jl.

The problem with the huge GeoIO dependency is that precompilation does not really help in CI where everything is recompiled again from scratch (we use GitLab at work with a self hosted instance so I do not have access to all the julia actions on github that might help with caching).

As a side comment, I remember that our precompilation time in GeoIO.jl increased massively with the addition of one single format (maybe it was geoparquet?). The issue #23 needs to be investigated by someone with more time and compiler skills. Perhaps, solving the issue can solve your huge CI times as a side effect.

I added the full copyright from this repo in the comment at the beginning of the file, let me know if this is sufficient or if I have to handle the copyright differently.

It is perfectly fine :) We are happy that you can find your way into the source code and adapt it to your needs.

EDIT: @juliohm Regarding how this differs from what is available in GeoArtifacts.NaturalEarth, not much in the raw content but I do export functionality in CountriesBorders to allow easy selection of specific countries and downfiltering based on region/name.

That would be super nice to have as a feature in GeoArtifacts.jl for example. This convenience options can help other users in the future. If you have them isolated, and could easily port in your free time, that would be great.

Will close the issue, but feel free to continue the discussion here.

from geoio.jl.

I have been thinking about checking how difficult or impactful on performance using CoordRefSystem internally would be so I might come again to open issues or ask questions in other packages of JuliaEarth.

That would be lovely. CoordRefSystems.jl unlocks various paths of innovation that were previously locked by PROJ. Feedback and improvements are very welcome.

Overall I think you are doing a very commendable effort into trying to harmonize these fields in Julia and I would like to re-use and contribute back to the ecosystem when possible :)

Appreciate your kind words. Despite the stubbornness of very few people, we are seeing more community members sharing our long-term vision.

from geoio.jl.