juliageo / gadm.jl Goto Github PK

View Code? Open in Web Editor NEW

38.0 38.0 4.0 144 KB

A Julia package for obtaining geographical data from the GADM dataset

Home Page: https://gadm.org

License: MIT License

Julia 100.00%

gadm geography julia julia-package package

gadm.jl's People

Contributors

Stargazers

Watchers

Forkers

zerefwayne clarohenrique rafaqz rcqls

gadm.jl's Issues

Always return Multipolygon from get function

Poster on JuliaCon without mention to package authors

@zerefwayne someone is presenting a poster on the GADM.jl package in this year's JuliaCon, are you aware of it? https://pretalx.com/juliacon2021/talk/KAJGA9/ The abstract doesn't mention your name.

Activate Travis CI + Codecov

@visr is Travis CI activated in JuliaGeo? The badges are pointing to an inactive build.

Use `geometry` rather than `geom`

GeoInterface.jl defaults to tables using geometry as the geometry column name. It would be nice if GADM.jl tables just worked in e.g. Rasters.jl without manually specifying the geometry column or getting them out.

See:
https://github.com/JuliaGeo/GeoInterface.jl/blob/main/src/interface.jl#L47

This default can be overridden with a custom table type in GeoInterface.geometrycolumns. But for NamedTuple tables
that requires type pyracy, so using the default column name is the way to get compatibility.

Postprocess Geopackage dataset into a tree

Update to ArchGDAL.jl v0.7

@zerefwayne any chance you have some free time to try update this package to the latest stable version of ArchGDAL.jl?

Here is a relevant discussion: yeesian/ArchGDAL.jl#210

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

UC Davis is extremely flaky

GeoMakie uses GADM in some examples, and routinely has issues with the UC Davis servers being down.

Are there any other URLs that serve GADM, that could be used as alternate sources?

DataDeps warning when running `get`

GADM.get("JPN") # first run

yields a warning:

┌ Warning: Checksum not provided, add to the Datadep Registration the following hash line
│   hash = "a7d0955dfd9215d6b21cff3f5b7bf09b02c3b64290e7b6ac1b357b7789f45e03"
└ @ DataDeps ~/.julia/packages/DataDeps/Y2lje/src/verification.jl:44

I get similar errors for every other country. This should probably be resolved in the registration block here:

GADM.jl/src/GADM.jl

Lines 64 to 69 in 606b61a

    
           register(DataDep(ID, 
        
               "Geographic data for country $country provided by the https://gadm.org project.", 
        
               "$route/$filename", 
        
               post_fetch_method=postfetch 
        
           )) 
        
           @datadep_str ID

get function doesn't work

Here is what I get when I try the master branch:

ERROR: IOError: could not spawn `unzip -x /home/juliohm/.julia/datadeps/GADM_BRA/gadm36_BRA_gpkg.zip -d /home/juliohm/.julia/datadeps/GADM_BRA`: no such file or directory (ENOENT)
Stacktrace:
 [1] _spawn_primitive(::String, ::Cmd, ::Array{Any,1}) at ./process.jl:99
 [2] #585 at ./process.jl:112 [inlined]
 [3] setup_stdios(::Base.var"#585#586"{Cmd}, ::Array{Any,1}) at ./process.jl:196
 [4] _spawn at ./process.jl:111 [inlined]
 [5] run(::Cmd; wait::Bool) at ./process.jl:439
 [6] run at ./process.jl:438 [inlined]
 [7] unpack(::String; keep_originals::Bool) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/post_fetch_helpers.jl:63
 [8] unpack at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/post_fetch_helpers.jl:63 [inlined]
 [9] #16 at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution_automatic.jl:122 [inlined]
 [10] cd(::DataDeps.var"#16#17"{typeof(DataDeps.unpack),String}, ::String) at ./file.jl:104
 [11] run_post_fetch(::typeof(DataDeps.unpack), ::String) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution_automatic.jl:119
 [12] download(::DataDeps.DataDep{Nothing,String,typeof(DataDeps.fetch_default),typeof(DataDeps.unpack)}, ::String; remotepath::String, i_accept_the_terms_of_use::Nothing, skip_checksum::Bool) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution_automatic.jl:84
 [13] download at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution_automatic.jl:70 [inlined]
 [14] handle_missing at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution_automatic.jl:10 [inlined]
 [15] _resolve(::DataDeps.DataDep{Nothing,String,typeof(DataDeps.fetch_default),typeof(DataDeps.unpack)}, ::String) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution.jl:83
 [16] resolve(::DataDeps.DataDep{Nothing,String,typeof(DataDeps.fetch_default),typeof(DataDeps.unpack)}, ::String, ::String) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution.jl:29
 [17] resolve(::String, ::String, ::String) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution.jl:54
 [18] resolve at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution.jl:73 [inlined]
 [19] download(::String) at /home/juliohm/.julia/packages/Geography/IdS6b/src/download.jl:36
 [20] get(::String) at /home/juliohm/.julia/packages/Geography/IdS6b/src/polygon.jl:87
 [21] top-level scope at REPL[8]:1
caused by [exception 1]
KeyError: key "GADM_BRA" not found
Stacktrace:
 [1] getindex at ./dict.jl:467 [inlined]
 [2] resolve(::String, ::String, ::String) at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution.jl:54
 [3] resolve at /home/juliohm/.julia/packages/DataDeps/PcUPE/src/resolution.jl:73 [inlined]
 [4] download(::String) at /home/juliohm/.julia/packages/Geography/IdS6b/src/download.jl:26
 [5] get(::String) at /home/juliohm/.julia/packages/Geography/IdS6b/src/polygon.jl:87
 [6] top-level scope at REPL[8]:1

Explore open datasets of geographic borders (countries, states, cities)

Write a README.md file with examples

Try to follow the convention of this README: https://github.com/JuliaEarth/ExtremeStats.jl

An installation and an usage section at least. 👍

ArchGDAL.jl is not building on GitHub Actions

The tests in GADM.jl are failing because ArchGDAL.jl is not compiling.

Is there any special config for GDAL on GitHub Actions?

[GPKG] Add function to process gpkg using ArchGDAL package

Set ENV["DATADEPS_ALWAYS_ACCEPT"] = true in init()

Beginners always fall into this trap of trying to download a geometry and failing to see the confirmation in the prompt. We should probably set the environment variable in the init() of the package so that downloads are always done without user interaction.

[Test] Add test suite for download.jl

Replace ISO 3166-2 Alpha 3 to Alpha 2 codes

GADM Dataset follows the standard of Alpha 2 codes, i.e. India code will be "IN" instead of "IND"

GADM is down... and maybe the error could be more informative.

Running the readme command

 GADM.coordinates("IND")

the prompt asks to download the data twice, and does start... but ends with a 403 error.

Turns out this is because GADM is down: this file wont load in the browser either.
https://data.biogeo.ucdavis.edu/data/gadm3.6/gpkg/gadm36_IND_gpkg.zip

Seems ucdavis doesn't run reliable servers
https://stackoverflow.com/questions/58265951/http-status-was-403-forbidden-when-trying-to-download-spatial-polygon-datafram

Its a minor point, but it's the error output is pretty long, with an apparently duplicated stack trace. I don't handle this possibility in RasterDataSources.jl either, but it seems like we should account for it.

We could maybe have some `The host site appears to be down" message and a url to the actual file to test in the browser, and reassure people its not a code problem, and that waiting or contacting GADM is all they can do.

Here's the current stack trace:

┌ Info: Downloading
│   source = "https://biogeo.ucdavis.edu/data/gadm3.6/gpkg/gadm36_IND_gpkg.zip"
│   dest = "/home/raf/.julia/datadeps/GADM_IND/gadm36_IND_gpkg.zip"
│   progress = 1.0
│   time_taken = "0.0 s"
│   time_remaining = "0.0 s"
│   average_speed = "∞ B/s"
│   downloaded = "289 bytes"
│   remaining = "0 bytes"
└   total = "289 bytes"
ERROR: HTTP.ExceptionRequest.StatusError(403, "GET", "/data/gadm3.6/gpkg/gadm36_IND_gpkg.zip", HTTP.Messages.Response:
"""
HTTP/1.1 403 Forbidden
Date: Sun, 03 Oct 2021 09:43:07 GMT
Server: Apache/2.4.18 (Ubuntu)
Content-Length: 289
Content-Type: text/html; charset=iso-8859-1

""")
Stacktrace:
  [1] request(::Type{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}, ::URIs.URI, ::Vararg{Any, N} where N; kw::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:iofunction, :reached_redirect_limit), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Bool}}})
    @ HTTP.ExceptionRequest ~/.julia/packages/HTTP/qszg7/src/ExceptionRequest.jl:22
  [2] (::Base.var"#70#72"{Base.var"#70#71#73"{ExponentialBackOff, HTTP.RetryRequest.var"#2#3"{Bool, HTTP.Messages.Request}, typeof(HTTP.request)}})(::Type, ::Vararg{Any, N} where N; kwargs::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:iofunction, :reached_redirect_limit), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Bool}}})
    @ Base ./error.jl:301
  [3] #request#1
    @ ~/.julia/packages/HTTP/qszg7/src/RetryRequest.jl:44 [inlined]
  [4] request(::Type{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}, method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing; http_version::VersionNumber, target::String, parent::HTTP.Messages.Response, iofunction::Function, kw::Base.Iterators.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:reached_redirect_limit,), Tuple{Bool}}})
    @ HTTP.MessageRequest ~/.julia/packages/HTTP/qszg7/src/MessageRequest.jl:66
  [5] request(::Type{HTTP.BasicAuthRequest.BasicAuthLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}}, method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing; kw::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:reached_redirect_limit, :iofunction, :parent), Tuple{Bool, HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, HTTP.Messages.Response}}})
    @ HTTP.BasicAuthRequest ~/.julia/packages/HTTP/qszg7/src/BasicAuthRequest.jl:28
  [6] request(::Type{HTTP.RedirectRequest.RedirectLayer{HTTP.BasicAuthRequest.BasicAuthLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}}}, method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing; redirect_limit::Int64, forwardheaders::Bool, kw::Base.Iterators.Pairs{Symbol, HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Tuple{Symbol}, NamedTuple{(:iofunction,), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}}}})
    @ HTTP.RedirectRequest ~/.julia/packages/HTTP/qszg7/src/RedirectRequest.jl:28
  [7] #request#1
    @ ~/.julia/packages/HTTP/qszg7/src/TopRequest.jl:15 [inlined]
  [8] request(method::String, url::String, h::Vector{Pair{SubString{String}, SubString{String}}}, b::Nothing; headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing, query::Nothing, kw::Base.Iterators.Pairs{Symbol, HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Tuple{Symbol}, NamedTuple{(:iofunction,), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}}}})
    @ HTTP ~/.julia/packages/HTTP/qszg7/src/HTTP.jl:330
  [9] #open#11
    @ ~/.julia/packages/HTTP/qszg7/src/HTTP.jl:364 [inlined]
 [10] open
    @ ~/.julia/packages/HTTP/qszg7/src/HTTP.jl:364 [inlined]
 [11] #download#27
    @ ~/.julia/packages/HTTP/qszg7/src/download.jl:110 [inlined]
 [12] fetch_http(remotepath::String, localdir::String; update_period::Float32)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/fetch_helpers.jl:80
 [13] fetch_http
    @ ~/.julia/packages/DataDeps/ooWXe/src/fetch_helpers.jl:79 [inlined]
 [14] fetch_default(remotepath::String, localdir::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/fetch_helpers.jl:33
 [15] run_fetch
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:99 [inlined]
 [16] download(datadep::DataDeps.DataDep{Nothing, String, typeof(DataDeps.fetch_default), typeof(DataDeps.unpack)}, localdir::String; remotepath::String, i_accept_the_terms_of_use::Nothing, skip_checksum::Bool)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:78
 [17] download
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:70 [inlined]
 [18] handle_missing
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:10 [inlined]
 [19] _resolve(datadep::DataDeps.DataDep{Nothing, String, typeof(DataDeps.fetch_default), typeof(DataDeps.unpack)}, calling_filepath::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:83
 [20] resolve(datadep::DataDeps.DataDep{Nothing, String, typeof(DataDeps.fetch_default), typeof(DataDeps.unpack)}, inner_filepath::String, calling_filepath::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:29
 [21] resolve(datadep_name::String, inner_filepath::String, calling_filepath::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:54
 [22] resolve
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:73 [inlined]
 [23] download(country::String)
    @ GADM ~/.julia/packages/GADM/OFrEW/src/GADM.jl:36
 [24] |>
    @ ./operators.jl:858 [inlined]
 [25] getdataset
    @ ~/.julia/packages/GADM/OFrEW/src/GADM.jl:63 [inlined]
 [26] get(::String; children::Bool)
    @ GADM ~/.julia/packages/GADM/OFrEW/src/GADM.jl:104
 [27] get
    @ ~/.julia/packages/GADM/OFrEW/src/GADM.jl:104 [inlined]
 [28] coordinates(::String)
    @ GADM ~/.julia/packages/GADM/OFrEW/src/GADM.jl:160
 [29] top-level scope
    @ REPL[10]:1

caused by: HTTP.ExceptionRequest.StatusError(403, "GET", "/data/gadm3.6/gpkg/gadm36_IND_gpkg.zip", HTTP.Messages.Response:
"""
HTTP/1.1 403 Forbidden
Date: Sun, 03 Oct 2021 09:42:53 GMT
Server: Apache/2.4.18 (Ubuntu)
Content-Length: 289
Content-Type: text/html; charset=iso-8859-1

""")
Stacktrace:
  [1] request(::Type{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}, ::URIs.URI, ::Vararg{Any, N} where N; kw::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:iofunction, :reached_redirect_limit), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Bool}}})
    @ HTTP.ExceptionRequest ~/.julia/packages/HTTP/qszg7/src/ExceptionRequest.jl:22
  [2] (::Base.var"#70#72"{Base.var"#70#71#73"{ExponentialBackOff, HTTP.RetryRequest.var"#2#3"{Bool, HTTP.Messages.Request}, typeof(HTTP.request)}})(::Type, ::Vararg{Any, N} where N; kwargs::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol}, NamedTuple{(:iofunction, :reached_redirect_limit), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Bool}}})
    @ Base ./error.jl:301
  [3] #request#1
    @ ~/.julia/packages/HTTP/qszg7/src/RetryRequest.jl:44 [inlined]
  [4] request(::Type{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}, method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing; http_version::VersionNumber, target::String, parent::HTTP.Messages.Response, iofunction::Function, kw::Base.Iterators.Pairs{Symbol, Bool, Tuple{Symbol}, NamedTuple{(:reached_redirect_limit,), Tuple{Bool}}})
    @ HTTP.MessageRequest ~/.julia/packages/HTTP/qszg7/src/MessageRequest.jl:66
  [5] request(::Type{HTTP.BasicAuthRequest.BasicAuthLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}}, method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing; kw::Base.Iterators.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:reached_redirect_limit, :iofunction, :parent), Tuple{Bool, HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, HTTP.Messages.Response}}})
    @ HTTP.BasicAuthRequest ~/.julia/packages/HTTP/qszg7/src/BasicAuthRequest.jl:28
  [6] request(::Type{HTTP.RedirectRequest.RedirectLayer{HTTP.BasicAuthRequest.BasicAuthLayer{HTTP.MessageRequest.MessageLayer{HTTP.RetryRequest.RetryLayer{HTTP.ExceptionRequest.ExceptionLayer{HTTP.ConnectionRequest.ConnectionPoolLayer{HTTP.StreamRequest.StreamLayer{Union{}}}}}}}}}, method::String, url::URIs.URI, headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing; redirect_limit::Int64, forwardheaders::Bool, kw::Base.Iterators.Pairs{Symbol, HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Tuple{Symbol}, NamedTuple{(:iofunction,), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}}}})
    @ HTTP.RedirectRequest ~/.julia/packages/HTTP/qszg7/src/RedirectRequest.jl:28
  [7] #request#1
    @ ~/.julia/packages/HTTP/qszg7/src/TopRequest.jl:15 [inlined]
  [8] request(method::String, url::String, h::Vector{Pair{SubString{String}, SubString{String}}}, b::Nothing; headers::Vector{Pair{SubString{String}, SubString{String}}}, body::Nothing, query::Nothing, kw::Base.Iterators.Pairs{Symbol, HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}, Tuple{Symbol}, NamedTuple{(:iofunction,), Tuple{HTTP.var"#28#34"{Float32, String, String, Vector{String}, HTTP.var"#format_bytes_per_second#33"{HTTP.var"#format_bytes#31"}, HTTP.var"#format_seconds#32", HTTP.var"#format_bytes#31", HTTP.var"#format_progress#30"}}}})
    @ HTTP ~/.julia/packages/HTTP/qszg7/src/HTTP.jl:330
  [9] #open#11
    @ ~/.julia/packages/HTTP/qszg7/src/HTTP.jl:364 [inlined]
 [10] open
    @ ~/.julia/packages/HTTP/qszg7/src/HTTP.jl:364 [inlined]
 [11] #download#27
    @ ~/.julia/packages/HTTP/qszg7/src/download.jl:110 [inlined]
 [12] fetch_http(remotepath::String, localdir::String; update_period::Float32)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/fetch_helpers.jl:80
 [13] fetch_http
    @ ~/.julia/packages/DataDeps/ooWXe/src/fetch_helpers.jl:79 [inlined]
 [14] fetch_default(remotepath::String, localdir::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/fetch_helpers.jl:33
 [15] run_fetch
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:99 [inlined]
 [16] download(datadep::DataDeps.DataDep{Nothing, String, typeof(DataDeps.fetch_default), typeof(DataDeps.unpack)}, localdir::String; remotepath::String, i_accept_the_terms_of_use::Nothing, skip_checksum::Bool)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:78
 [17] download
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:70 [inlined]
 [18] handle_missing
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution_automatic.jl:10 [inlined]
 [19] _resolve(datadep::DataDeps.DataDep{Nothing, String, typeof(DataDeps.fetch_default), typeof(DataDeps.unpack)}, calling_filepath::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:83
 [20] resolve(datadep::DataDeps.DataDep{Nothing, String, typeof(DataDeps.fetch_default), typeof(DataDeps.unpack)}, inner_filepath::String, calling_filepath::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:29
 [21] resolve(datadep_name::String, inner_filepath::String, calling_filepath::String)
    @ DataDeps ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:54
 [22] resolve
    @ ~/.julia/packages/DataDeps/ooWXe/src/resolution.jl:73 [inlined]
 [23] download(country::String)
    @ GADM ~/.julia/packages/GADM/OFrEW/src/GADM.jl:28
 [24] |>
    @ ./operators.jl:858 [inlined]
 [25] getdataset
    @ ~/.julia/packages/GADM/OFrEW/src/GADM.jl:63 [inlined]
 [26] get(::String; children::Bool)
    @ GADM ~/.julia/packages/GADM/OFrEW/src/GADM.jl:104
 [27] get
    @ ~/.julia/packages/GADM/OFrEW/src/GADM.jl:104 [inlined]
 [28] coordinates(::String)
    @ GADM ~/.julia/packages/GADM/OFrEW/src/GADM.jl:160
 [29] top-level scope
    @ REPL[10]:1

API discussion

If I understand correctly, this API always returns a single geometry:

get("IND")
get("IND", "Uttar Pradesh")
get("IND", "Uttar Pradesh", "Lucknow")

For a lot of use cases this is no doubt very useful, but am I correct that this does not allow me to get all of India's admin 1 level polygons at once? I imaging it would be useful to specify the level. Although that would no longer always return a single geometry, so perhaps it would be better to put that in a separate function.

Related, what do you think about returning a table instead, using this functionality: https://yeesian.com/ArchGDAL.jl/dev/tables/
That way, the attributes, which often contain useful metadata, would be included as well.

I guess this could be two separate functions, one like get that returns a single geometry, and one that returns an entire layer as an ArchGDAL.Table?

Non-existing country code should give clearer error

When I use a wrong country code in the get function I get a cryptic 404 HTTP error and I think it would be helpful to give a more descriptive error, so that users know what they made wrong.

GADM.get("GER")

HTTP.Exceptions.StatusError(404, "GET", "/gadm/gadm4.1/gpkg/gadm41_GER.gpkg", HTTP.Messages.Response:
"""
HTTP/1.1 404 Not Found
Date: Thu, 01 Jun 2023 07:29:33 GMT
Server: Apache/2.4.52 (Ubuntu)
Content-Length: 282
Content-Type: text/html; charset=iso-8859-1

""")

Instead of this error I would have hoped for an error stating.
ArgumentError: Country code "GER" not found, please provide standard ISO 3 country codes.

Getting lower level regions (e.g. all counties of the US)

Is there a better way of getting all counties of the US than this?

country_code = "USA"
lev0, lev1 = GADM.get(country_code, children = true)
lev1_ids = lev1.NAME_1
mapreduce(vcat, lev1_ids) do id1
    _, lev2 = GADM.get(country_code, id1, children = true)
    lev2 |> DataFrame
end

It would be nice to have something like

GADM.get("USA", level = 2)

Add GADM_ prefix to downloaded folders

Quite a minor point, but I noticed that the country codes are directly placed under the datadeps folder:

c:\Users\visser_mn\.julia\datadeps\IND\
c:\Users\visser_mn\.julia\datadeps\VAT\

Perhaps it would be better to keep them together under a GADM directory, such that if we want we can more easily delete them, because they won't be mixed with other DataDeps data?

c:\Users\visser_mn\.julia\datadeps\GADM\IND\
c:\Users\visser_mn\.julia\datadeps\GADM\VAT\

Geography.get("IND", "UP", "LKO"): Returns the polygon of Lucknow District in State UP.

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

	register(DataDep(ID,
	"Geographic data for country $country provided by the https://gadm.org project.",
	"$route/$filename",
	post_fetch_method=postfetch
	))
	@datadep_str ID