geocompx / docker Goto Github PK
View Code? Open in Web Editor NEWDockerfiles for Geocomputation
Home Page: https://github.com/geocompx/docker/pkgs/container/docker
License: GNU General Public License v3.0
Dockerfiles for Geocomputation
Home Page: https://github.com/geocompx/docker/pkgs/container/docker
License: GNU General Public License v3.0
Solution: use ubuntugis stable ppa
Dailly is a bit intense.
As discussed with @Nowosad currently the Docker images are rebuilt every time one of us pushes to the repo. This is not ideal. Any ideas for getting a more automated system going? It's tricky because DockerHub only triggers builds when it detects commits in this repo I think, so I was thinking of a cron job on my computer, but that is not an ideal solution. Suggestions welcome.
Hot on the heels of #45
Not through pip
...
Context: https://github.com/rocker-org/rocker-versioned2/blob/master/scripts/install_python.sh
With the following message:
Dockerfile:48
--------------------
46 | RUN sh -c 'echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/qgis-archive-keyring.gpg] https://qgis.org/ubuntugis `lsb_release -c -s` main" > /etc/apt/sources.list.d/qgis.list'
47 | RUN apt-get update
48 | >>> RUN apt-get install -y qgis qgis-plugin-grass saga
49 | RUN wget -qO sagang_plugin.zip https://plugins.qgis.org/plugins/processing_saga_nextgen/version/0.0.7/download/
50 | RUN unzip -q sagang_plugin.zip -d /home/rstudio/.local/share/QGIS/QGIS3/profiles/default/python/plugins
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get install -y qgis qgis-plugin-grass saga" did not complete successfully: exit code: 100
Error: buildx failed with: ERROR: failed to solve: process "/bin/sh -c apt-get install -y qgis qgis-plugin-grass saga" did not complete successfully: exit code: 100
Source: https://github.com/geocompx/docker/actions/runs/6321057960/job/17164486628#step:8:644
Relevant lines:
Lines 48 to 50 in 00f4c90
Any ideas @jannes-m ?
This is an issue associated with images that build on geocompr/geocompr:minimal
:
#6 0.349 > remotes::install_github('geocompr/geocompkg', upgrade = TRUE, dependencies = TRUE)
#6 0.542 Skipping install of 'geocompkg' from a github remote, the SHA1 (6922a37b) has not changed since last install.
#6 0.542 Use `force = TRUE` to force installation
#6 0.542 >
#6 0.542 >
#6 DONE 3.0s
Source: https://github.com/geocompr/docker/runs/3776530772#step:8:270
Heads-up @michaeldorman.
With this error:
Encountered problems while solving:
- nothing provides requested quarto
The environment can't be solved, aborting the operation
error libmamba Could not solve for environment specs
critical libmamba UnsatisfiableError
The command '/usr/local/bin/_dockerfile_shell.sh micromamba install -y -n base -f /tmp/env.yaml && micromamba clean --all --yes' returned a non-zero code: 1
I think I know why: geocompy is now in geocompx/geocompy not geocompx/py
Evidence: https://github.com/geocompx/docker/actions/runs/4501197328/jobs/7921300929#step:8:553
Seeing this message:
#8 53.84 Downloading Cartopy-0.18.0.tar.gz (14.4 MB)
#8 53.99 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.4/14.4 MB 92.6 MB/s eta 0:00:00
#8 54.39 Preparing metadata (setup.py): started
#8 54.63 Preparing metadata (setup.py): finished with status 'error'
#8 54.64 error: subprocess-exited-with-error
#8 54.64
#8 54.64 × python setup.py egg_info did not run successfully.
#8 54.64 │ exit code: 1
#8 54.64 ╰─> [13 lines of output]
#8 54.64 Traceback (most recent call last):
#8 54.64 File "/tmp/pip-install-xpr0hdls/cartopy_f8366d3bf5f44cd6b51d81b49e18c0f5/setup.py", line 43, in <module>
#8 54.64 import numpy as np
#8 54.64 ModuleNotFoundError: No module named 'numpy'
#8 54.64
#8 54.64 During handling of the above exception, another exception occurred:
#8 54.64
#8 54.64 Traceback (most recent call last):
#8 54.64 File "<string>", line 2, in <module>
#8 54.64 File "<pip-setuptools-caller>", line 34, in <module>
#8 54.64 File "/tmp/pip-install-xpr0hdls/cartopy_f8366d3bf5f44cd6b51d81b49e18c0f5/setup.py", line 45, in <module>
#8 54.64 raise ImportError('NumPy 1.10+ is required to install cartopy.')
#8 54.64 ImportError: NumPy 1.10+ is required to install cartopy.
#8 54.64 [end of output]
Source: https://github.com/geocompr/docker/runs/7054120092?check_suite_focus=true#step:8:736 Any ideas welcome, call for help in the dark @martinfleis!
E.g. as can be seen here for proj: https://launchpad.net/~ubuntugis/+archive/ubuntu/ubuntugis-unstable?field.series_filter=focal
Plan: remove these Docker images.
As shown below:
Source: https://github.com/geocompx/docker/actions/workflows/suggests.yml
Related: geocompx/geocompr#1049
Can we add GRASS, SAGA and qgisprocess to our geocompr:qgis image? This way we could harness the full potential of QGIS, and it would be also rather helpful for writing the qgisprocess vignette (see r-spatial/qgisprocess#16). For now, I have simply adjusted and rebuild your Dockerfile locally. Thinking it over again, yes, it is a good idea to add qgisprocess but for writing the vignette it makes more sense to mount my local qgisprocess repo. Hence, we would have to adjust the Dockerfile as follows (I can put in a PR if you want me to):
FROM geocompr/geocompr:qgis
RUN apt-get update \
&& apt-get install --no-install-recommends --no-install-suggests --allow-unauthenticated -y \
qgis-plugin-grass \
saga
RUN env QT_QPA_PLATFORM='offscreen' qgis_process
RUN echo "[PythonPlugins]\nprocessing=true" >> root/.local/share/QGIS/QGIS3/profiles/default/QGIS/QGIS3.ini
RUN R -e "remotes::install_github('paleolimbot/qgisprocess')"
The penultimate line is necessary to enable 3rd party providers when running qgis_process in a headless state (see qgis/QGIS#37989). And we need to execute the line before (RUN env QT_QPA_PLATFORM='offscreen' qgis_process) in order to create the QGIS3.ini file (maybe a bad hack, but it works).
FYI @paleolimbot
As documented here: https://pythonspeed.com/articles/activate-conda-dockerfile/
With upstream containers building first.
Context: geocompx/geocompy#52 and #26
For some reason. Logs: https://github.com/geocompr/docker/runs/5431901131?check_suite_focus=true#step:8:695
Heads-up @anitagraser plan for now is to remove movingpandas and add later. I'll keep this issue open to track it.
I may have made a silly mistake, quite likely as currently my Python dev skills amount to guesswork!
With the current set-up we don't do any tagging of different versions. However it would improve reproducibility if we could add tags making it easier for people to get images produced at a particular data/release. The action on which our Docker actions are based enable tagging but I wasn't sure how best to implement this https://github.com/docker/build-push-action/blob/master/docs/advanced/tags-labels.md
Any comments/suggestions on this very welcome.
@Robinlovelace do I understand correctly that the geocompr images are based on rocker-org/geospatial? If so, https://github.com/rocker-org/geospatial uses very old R version and has not been updated in ~2 years. Should we do something with it?
To include extras.
Or similar, see error messages here:
#10 757.2 Failed to build cartopy pandas
#10 757.2 ERROR: Could not build wheels for cartopy, which is required to install pyproject.toml-based projects
------
Dockerfile:13
--------------------
11 | RUN pip3 install rasterio rioxarray netcdf4 h5netcdf
12 | # Install Python packages for visualisation
13 | >>> RUN pip3 install jupyter matplotlib hvplot pyviz geoviews
14 | # Install quarto
15 | RUN curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | sudo dd of=/etc/apt/trusted.gpg.d/githubcli-archive-keyring.gpg
--------------------
error: failed to solve: process "/bin/sh -c pip3 install jupyter matplotlib hvplot pyviz geoviews" did not complete successfully: exit code: 1
Error: buildx failed with: error: failed to solve: process "/bin/sh -c pip3 install jupyter matplotlib hvplot pyviz geoviews" did not complete successfully: exit code: 1
https://github.com/geocompr/docker/runs/5438444509?check_suite_focus=true#step:8:6181
Any ideas @anitagraser or anyone?
The README is R focussed but in fact we now have the Python images.
I hit issues with sf
installation but this can be worked around following guidance from the Rocker maintainers here: rocker-org/rocker-versioned2#631
Hi @Robinlovelace -- are there any plans to rename images from geocompr to geocompx?
I need this for my own purposes but will be useful for others I think.
As discussed with @benz0li, we could use the b-data
stack throughout, with more consistency and good R/Python support. First we would need to think about similarities/differences and pros/cons, leaving this as an open question, I'm a bit out of my depth, so any advice / PRs welcome : )
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.