eco4cast / gefs4cast Goto Github PK
View Code? Open in Web Editor NEWHome Page: https://projects.ecoforecast.org/gefs4cast
License: Other
gefs4cast's People
Contributors
Stargazers
Watchers
gefs4cast's Issues
Can't find `bash` at `/usr/bin/bash`
I get an error running gefs_cog():
library(gefs4cast)
gefs_cog()
#> Error in c("process_initialize(self, private, command, args, stdin, stdout, ", : ! Native call to `processx_exec` failed
#> Caused by error in `chain_call(c_processx_exec, command, c(command, args), pty, pty_options, …`:
#> ! cannot start processx process '/usr/bin/bash' (system error 2, No such file or directory) @unix/processx.c:613 (processx_exec)Created on 2022-12-20 with reprex v2.0.2
Similar to #5, it looks like there are some assumptions made in the package about where bash is. For me (macOs 13.1) it's at /bin/bash, not /usr/bin/bash.
If I run debugonce(gefs4cast:::gdal_download) and replace p <- processx::run("/usr/bin/bash", shell) with p <- processx::run("bash", shell) I think it works fine though.
site_id partitioning
Should we add the option for a site_id partition in processed parquet files? Without having site_id it is hard to add new sites to an existing catalog of downloaded forecasts.
`gefs_cog()` doesn't seem to save any output
My computer is definitely working hard, but nothing ends up being saved to disk by gefs_cog(). noaa_gefs() does save files out given the same arguments.
library(gefs4cast)
gefs_cog(
dest = "gefs",
date = Sys.Date()-1,
max_horizon = 2
)
list.files("gefs", recursive = TRUE, include.dirs = TRUE)
#> [1] "gefs.20221220"
noaa_gefs(
dest = "noaa",
date = Sys.Date()-1,
max_horizon = 2
)
#> date: 2022-12-20
#> Error: IOError: When initiating multiple part upload for key 'noaa/gefs-v12/stage1/0/2022-12-20/part-0.parquet' in bucket 'drivers': AWS Error ACCESS_DENIED during CreateMultipartUpload operation: Access Denied.
list.files("noaa", recursive = TRUE, include.dirs = TRUE)
#> [1] "gefs.20221220"
#> [2] "gefs.20221220/gec00.t00z.pgrb2a.0p50.f000.tif"
#> [3] "gefs.20221220/gep01.t00z.pgrb2a.0p50.f000.tif"
#> [4] "gefs.20221220/gep02.t00z.pgrb2a.0p50.f000.tif"
#> [5] "gefs.20221220/gep03.t00z.pgrb2a.0p50.f000.tif"
#> [6] "gefs.20221220/gep04.t00z.pgrb2a.0p50.f000.tif"
#> [7] "gefs.20221220/gep05.t00z.pgrb2a.0p50.f000.tif"
#> [8] "gefs.20221220/gep06.t00z.pgrb2a.0p50.f000.tif"
#> [9] "gefs.20221220/gep07.t00z.pgrb2a.0p50.f000.tif"
#> [10] "gefs.20221220/gep08.t00z.pgrb2a.0p50.f000.tif"
#> [11] "gefs.20221220/gep09.t00z.pgrb2a.0p50.f000.tif"
#> [12] "gefs.20221220/gep10.t00z.pgrb2a.0p50.f000.tif"
#> [13] "gefs.20221220/gep11.t00z.pgrb2a.0p50.f000.tif"
#> [14] "gefs.20221220/gep12.t00z.pgrb2a.0p50.f000.tif"
#> [15] "gefs.20221220/gep13.t00z.pgrb2a.0p50.f000.tif"
#> [16] "gefs.20221220/gep14.t00z.pgrb2a.0p50.f000.tif"
#> [17] "gefs.20221220/gep15.t00z.pgrb2a.0p50.f000.tif"
#> [18] "gefs.20221220/gep16.t00z.pgrb2a.0p50.f000.tif"
#> [19] "gefs.20221220/gep17.t00z.pgrb2a.0p50.f000.tif"
#> [20] "gefs.20221220/gep18.t00z.pgrb2a.0p50.f000.tif"
#> [21] "gefs.20221220/gep19.t00z.pgrb2a.0p50.f000.tif"
#> [22] "gefs.20221220/gep20.t00z.pgrb2a.0p50.f000.tif"
#> [23] "gefs.20221220/gep21.t00z.pgrb2a.0p50.f000.tif"
#> [24] "gefs.20221220/gep22.t00z.pgrb2a.0p50.f000.tif"
#> [25] "gefs.20221220/gep23.t00z.pgrb2a.0p50.f000.tif"
#> [26] "gefs.20221220/gep24.t00z.pgrb2a.0p50.f000.tif"
#> [27] "gefs.20221220/gep25.t00z.pgrb2a.0p50.f000.tif"
#> [28] "gefs.20221220/gep26.t00z.pgrb2a.0p50.f000.tif"
#> [29] "gefs.20221220/gep27.t00z.pgrb2a.0p50.f000.tif"
#> [30] "gefs.20221220/gep28.t00z.pgrb2a.0p50.f000.tif"
#> [31] "gefs.20221220/gep29.t00z.pgrb2a.0p50.f000.tif"
#> [32] "gefs.20221220/gep30.t00z.pgrb2a.0p50.f000.tif"Created on 2022-12-21 with reprex v2.0.2
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.2.0 (2022-04-22)
#> os macOS Big Sur/Monterey 10.16
#> system x86_64, darwin17.0
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz America/New_York
#> date 2022-12-21
#> pandoc 2.19.2 @ /Applications/RStudio.app/Contents/Resources/app/quarto/bin/tools/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> ! package * version date (UTC) lib source
#> P arrow 10.0.0 2022-10-26 [?] CRAN (R 4.2.0)
#> P assertthat 0.2.1 2019-03-21 [?] CRAN (R 4.2.0)
#> P bit 4.0.4 2020-08-04 [?] CRAN (R 4.2.0)
#> P bit64 4.0.5 2020-08-30 [?] CRAN (R 4.2.0)
#> P cli 3.4.1 2022-09-23 [?] CRAN (R 4.2.0)
#> P codetools 0.2-18 2020-11-04 [3] CRAN (R 4.2.0)
#> P crayon 1.5.2 2022-09-29 [?] CRAN (R 4.2.0)
#> P curl 4.3.3 2022-10-06 [?] CRAN (R 4.2.0)
#> P DBI 1.1.3 2022-06-18 [?] CRAN (R 4.2.0)
#> P digest 0.6.29 2021-12-01 [?] CRAN (R 4.2.0)
#> P dplyr 1.0.10 2022-09-01 [?] CRAN (R 4.2.0)
#> P ellipsis 0.3.2 2021-04-29 [?] CRAN (R 4.2.0)
#> P evaluate 0.17 2022-10-07 [?] CRAN (R 4.2.0)
#> P fansi 1.0.3 2022-03-24 [?] CRAN (R 4.2.0)
#> P fastmap 1.1.0 2021-01-25 [?] CRAN (R 4.2.0)
#> P fs 1.5.2 2021-12-08 [?] CRAN (R 4.2.0)
#> P gefs4cast * 0.0.1 2022-12-20 [?] Github (eco4cast/gefs4cast@10a5f0b)
#> P generics 0.1.3 2022-07-05 [?] CRAN (R 4.2.0)
#> P glue 1.6.2 2022-02-24 [?] CRAN (R 4.2.0)
#> P highr 0.9 2021-04-16 [?] CRAN (R 4.2.0)
#> P hms 1.1.2 2022-08-19 [?] CRAN (R 4.2.0)
#> P htmltools 0.5.3 2022-07-18 [?] CRAN (R 4.2.0)
#> P knitr 1.40 2022-08-24 [?] CRAN (R 4.2.0)
#> P lifecycle 1.0.3 2022-10-07 [?] CRAN (R 4.2.0)
#> P lubridate 1.9.0 2022-11-06 [?] CRAN (R 4.2.0)
#> P magrittr 2.0.3 2022-03-30 [?] CRAN (R 4.2.0)
#> P pillar 1.8.1 2022-08-19 [?] CRAN (R 4.2.0)
#> P pkgconfig 2.0.3 2019-09-22 [?] CRAN (R 4.2.0)
#> P processx 3.7.0 2022-07-07 [?] CRAN (R 4.2.0)
#> P ps 1.7.1 2022-06-18 [?] CRAN (R 4.2.0)
#> P purrr 0.3.5 2022-10-06 [?] CRAN (R 4.2.0)
#> R.cache 0.16.0 2022-07-21 [3] CRAN (R 4.2.0)
#> R.methodsS3 1.8.2 2022-06-13 [3] CRAN (R 4.2.0)
#> R.oo 1.25.0 2022-06-12 [3] CRAN (R 4.2.0)
#> R.utils 2.12.2 2022-11-11 [3] CRAN (R 4.2.0)
#> P R6 2.5.1 2021-08-19 [?] CRAN (R 4.2.0)
#> P Rcpp 1.0.9 2022-07-08 [?] CRAN (R 4.2.0)
#> P readr 2.1.3 2022-10-01 [?] CRAN (R 4.2.0)
#> P reprex 2.0.2 2022-08-17 [?] CRAN (R 4.2.0)
#> P rlang 1.0.6 2022-09-24 [?] CRAN (R 4.2.0)
#> P rmarkdown 2.17 2022-10-07 [?] CRAN (R 4.2.0)
#> P rstudioapi 0.14 2022-08-22 [?] CRAN (R 4.2.0)
#> sessioninfo 1.2.2 2021-12-06 [3] CRAN (R 4.2.0)
#> P stringi 1.7.8 2022-07-11 [?] CRAN (R 4.2.0)
#> P stringr 1.4.1 2022-08-20 [?] CRAN (R 4.2.0)
#> styler 1.8.1 2022-11-07 [3] CRAN (R 4.2.0)
#> P terra 1.6-41 2022-11-18 [?] CRAN (R 4.2.0)
#> P tibble 3.1.8 2022-07-22 [?] CRAN (R 4.2.0)
#> P tidyr 1.2.1 2022-09-08 [?] CRAN (R 4.2.0)
#> P tidyselect 1.2.0 2022-10-10 [?] CRAN (R 4.2.0)
#> P timechange 0.1.1 2022-11-04 [?] CRAN (R 4.2.0)
#> P tzdb 0.3.0 2022-03-28 [?] CRAN (R 4.2.0)
#> P utf8 1.2.2 2021-07-24 [?] CRAN (R 4.2.0)
#> P vctrs 0.5.0 2022-10-22 [?] CRAN (R 4.2.0)
#> P vroom 1.6.0 2022-09-30 [?] CRAN (R 4.2.0)
#> P withr 2.5.0 2022-03-03 [?] CRAN (R 4.2.0)
#> P xfun 0.34 2022-10-18 [?] CRAN (R 4.2.0)
#> P yaml 2.3.6 2022-10-18 [?] CRAN (R 4.2.0)
#>
#> [1] /Users/ericscott/Documents/GitHub/azmet-qaqc/renv/library/R-4.2/x86_64-apple-darwin17.0
#> [2] /Users/ericscott/Documents/GitHub/azmet-qaqc/renv/sandbox/R-4.2/x86_64-apple-darwin17.0/84ba8b13
#> [3] /Library/Frameworks/R.framework/Versions/4.2/Resources/library
#>
#> P ── Loaded and on-disk path mismatch.
#>
#> ──────────────────────────────────────────────────────────────────────────────detect gdal path
Assuming /usr/bin/local for the gdal bin path is pretty dumb: https://github.com/eco4cast/gefs4cast/blob/main/R/gdal_download.R#L12
Users have no way of realizing they can and must override this either.
Document this better, and add some non-stupid mechanism for guessing the right location. (consider relying on just respecting PATH? processx doesn't seem to like that...)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.