geonetwork / docker-geonetwork Goto Github PK
View Code? Open in Web Editor NEWOfficial Docker image repository for GeoNetwork
Official Docker image repository for GeoNetwork
database configuration is injected in scripts from docker only environment variables in a deploy script
However geonetwork already has a mechanism to inject environment parameters into the application, to prevent extra (contradicting) documentation, i recommend to use the default geonetwork injection mechanism
docker run -e jdbc.database=example...
Hello,
I'm try to harvest standard THREDDS services but whenever I try to do so, GeoNetwork is throwing a NullPointerException with the following stacktrace:
java.lang.NullPointerException
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.processServices(Harvester.java:1481)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:416)
at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor.process(AbstractHarvester.java:647)
at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:718)
at org.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)
at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)
The Webpage UI just shows:
Harvest error error
[]
This seems to be happening with any THREDDS service (my own or any publicly available).
Some examples: https://www.ncdc.noaa.gov/wct/data.php, https://tds.marine.rutgers.edu/thredds/met/wrf/catalog.html
My setup is running through docker-compose using the following YML:
version: '3.1'
services:
geonetwork:
image: geonetwork:3.12.2
restart: always
environment:
DATA_DIR: /var/lib/geonetwork
GEONETWORK_DB_NAME: /var/lib/geonetwork/db/gn
volumes:
- ./geonetwork_data:/var/lib/geonetwork
Does anyone have an idea of what could be causing this? Other harvesters seem to be working fine (i.e. harvesting other GeoNetworks, OGC SOS, etc...)
I was build this image on arm64 architecture and build successed .So I want to know we can add it to the https://hub.docker.com/u/arm64v8/ ?
Hello, we are using version 3.2.2, but we can not make geonetwork to upload files bigger than 100MB, after looking around we found a file: /usr/local/tomcat/webapps/geonetwork/WEB-INF/config-spring-geonetwork.xml where a code fragment must be placed attending some forums advices, but nothing happens:
<bean id="multipartResolver"
class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
<property name="maxUploadSize" value="1000000000"/>
<!-- set to 1GB -->
</bean>
After copying the new file we have restarted de container several times but the exceptions keeps rising on files greater than 100MB.
What others files must be changed or what docker configuration must be set in way to upload big files?
It seems that the property GEONETWORK_DB_CONNECTION_PROPERTIES is not used when using SQL server as the database. We added "encrypt=true;trustServerCertificate=true" to GEONETWORK_DB_CONNECTION_PROPERTIES but according to the logs it still tries to connect with "trustServerCertificate=false"
2024-04-04T11:41:43,255 WARN [geonetwork.databasemigration] - - Migration: Exception running migration for version: 4.4.3-0 subversion: SNAPSHOT. Cannot create PoolableConnectionFactory ("encrypt" property is set to "true" and "trustServerCertificate" property is set to "false" but the driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption: Error: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. ClientConnectionId:5726bea6-77a8-4256-ba88-69a1833ef109)
I'm not a Java developer by any means but shouldn't jdbc.connectionProperties
also be referenced in sqlserver.xml?
I have try to use the docker geonetwork:postgres with docker compose v3.
In my docker compose I have 2 service: one for my postgres database and one for my geonetwork.
I use depends_on and a command to verifiy that the database is working.
The command that I have use for that is
command: sh -c '/usr/bin/wait-for-it.sh myPostgresDocker:5432 -t 60 -- catalina.sh run'
When I used that command I see that the content of the Entrypoint is never executed, so the configuration for the database is never done. (the file jdbc.properties don't have properties like "jdbc.username")
For information , the script "wait-for-it.sh" is comming from that url "https://github.com/vishnubob/wait-for-it".
I don't know If I need to use another method to validate that my db is available or if a change have to be done to be able to use the docker compose v3 and to validate that a database is available before the configuration of the database.
For now I have change the command like that to solve the problem :
"sh -c '/usr/bin/wait-for-it.sh myPostgresDocker:5432 -t 60 -- /entrypoint.sh catalina.sh run'".
Currently when you access localhost:port you get some file-not-found error, you have to navigate to /geonetwork
to open the application
Instead would be better to install geonetwork at root, by:
ROOT.war
In case of context GeoNetwork can be deployed in /opt/webapps/geonetwork
Example of tomcat context:
<Context
docBase="/opt/webapps/geonetwork"
path=""
reloadable="false"
/>
From next major update Elastic will be essential component of GN, currently is optional, although offers nice extra capabilities. Would be interesting to add environment variables that connect geonetwork to an external (or orchestrated) elastic instance.
There are concerns raised here about the provenance of the code in OpenJDK 8 base images, both in terms of OpenJDK builds and older Debian distributions used as a base:
https://mail.openjdk.java.net/pipermail/jdk8u-dev/2019-May/009330.html
And certainly when we run a container security scan on the geonetwork:3.6.0 docker image we are seeing a lot of CVEs. Switching from a debian stretch to an alpine base image has eliminated most or all in other projects.
https://hub.docker.com/_/tomcat - this has 8.5.41-jre8-alpine, 8.5-jre8-alpine, 8-jre8-alpine, jre8-alpine, 8.5.41-alpine, 8.5-alpine, 8-alpine any of which you could consider as a base for the official geonetwork image and would improve the situation somewhat
Since gn.h2.db
is created in /usr/local/tomcat/webapps
it is lost every time the container is destroyed.
A workaround is to mount /usr/local/tomcat/webapps/gh.h2.db
as a volume, but the preferred method would be to store the database in the DATA_DIR
or in another place specified by an environment variable.
Using 3.12.8-postgres may have a bug in the Feedback configuration for sending emails, as it fails:
Sending the email to the following server failed : email-smtp.eu-west-1.amazonaws.com:2465. Could not connect to SMTP host: email-smtp.eu-west-1.amazonaws.com, port: 2465, response: -1
it also fails with 25, 587 or 2587, and with 465 or 2465
Any ideas?
Since v4.4 the individual environment variables for configuring Elasticsearch are obsolete. e.g. ES_HOST
, ES_PORT
, ES_PASSWORD
etc.
This is a regression for us since we rely on passing secrets to individual environment variables from AWS secrets manager/parameter store using the "value_from" directive. See https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar-ssm-paramstore.html#secrets-envvar-ssm-paramstore-update-container-definition
Since we now have to pass all Geonetwork settings in the one variable GN_CONFIG_PROPERTIES
it makes it necessary to store the secret for ES_PASSWORD
in plain-text in code, unless we store all the values for GN_CONFIG_PROPERTIES
as a secret too. However this becomes annoying as we ideally want to be able to look at the code to see what a particular environment is using.
Is there a good reason why these variables were deprecated?
Hi there,
I'm a bit worried, about how the best way to set another admin password when I create a derived docker container.
A lot of other docker images provide this kind of option to set those via env parameter (postgresql, wildfly, ...), but for the geonetwork I found no proper way to do this.
Is there a guideline / tutorial / pointer on how to do that?
now is hardcoded an image for use with postgres, maybe we can use an image where dbtype is a parameter, people can decide to use mysql, sqlserver, oracle or postgres
off course we have a challenge to download dynamically latest db driver (or include all?)
h2 could be the default value for dbtype, in that case we would only have a single docker image to maintain
There is no schema plugins and I have no idea how to add a new one from https://github.com/metadata101/iso19139.mcp
Hi,
I've followed the install documentation to persist data, and wanted to use graphicOverview in my catalog.
DATA_DIR=/var/lib/geonetwork_data -v /host/geonetwork-docker:/var/lib/geonetwork_data
I've copied thumbnail images to - /host/geonetwork-docker/data/Thumbnails/Geo_125k/XXXXX.jpg
And edited the record to the below which I thought would pick up the file but it's not working.
gmd:graphicOverview
gmd:MD_BrowseGraphic
gmd:fileName
gco:CharacterString/var/lib/geonetwork_data/data/Thumbnails/Geo_125k/XXXXX.jpg</gco:CharacterString>
</gmd:fileName>
gmd:fileDescription
gco:CharacterStringThumbnail</gco:CharacterString>
</gmd:fileDescription>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>
Am I misunderstanding how to use relative paths between docker and host volumes?
I try to create a docker composition so that geonetwork and geoserver connect to the same Postgres/PostGIS database.
So I use geonetwork:postgres
image as suggested in the doc to connect to a postgres docker container.
When starting the containers, I get several SQLException errors, and all of them end with:
Initialization of bean failed; nested exception is java.lang.RuntimeException:
java.sql.SQLException: Cannot create PoolableConnectionFactory
(The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.)
Here is my docker-compose.yml
v3.1 example file (without the geoserver container) so that you can reproduce the problem :
version: '3.1'
services:
geonetwork:
image: geonetwork:postgres
restart: always
ports:
- 8080:8080
command: bash -c 'while !</dev/tcp/db/5432; do sleep 1; done; sh /entrypoint.sh catalina.sh run'
environment:
DATA_DIR: /var/lib/geonetwork_data
POSTGRES_DB_HOST: db
POSTGRES_DB_PORT: 5432
POSTGRES_DB_NAME: ${POSTGRES_GN_DB}
POSTGRES_DB_USERNAME: ${POSTGRES_USER}
POSTGRES_DB_PASSWORD: ${POSTGRES_PASS}
volumes:
- geonetwork:/var/lib/geonetwork_data
db:
image: kartoza/postgis:13.0
volumes:
- geo-db-data:/var/lib/postgresql
ports:
- "25434:5432"
environment:
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER=${POSTGRES_USER}
- POSTGRES_PASS=${POSTGRES_PASS}
- POSTGRES_PASSWORD=${POSTGRES_PASS}
- ALLOW_IP_RANGE=${ALLOW_IP_RANGE}
restart: on-failure
healthcheck:
test: "exit 0"
volumes:
geonetwork:
geoserver-data:
geo-db-data:
Notes:
.env
file, but you can specify your own values.kartoza/postgis:13.0
because it is the one suggested by the geoserver docker documentation.By the way, if I try to set a database service with a different image (like postgis/postgis
), everything works fine. Yet, both images set up a postgres v13.0 database with a postgis v3.0.
I guess this is a matter of postgres configuration, with some differences between the images, but I don't know which ones.
Could you elaborate on what postgres/postgis versions are compatible with the geonetwork docker images, and maybe specify this in the documentation ?
Starting the Docker image fails with "Environment variable ES_HOST is mandatory". This variable is not mentioned in the Docker image documentation:
This command will start a debian-based container, running a Tomcat web server, with a geonetwork war deployed on the server:
docker run --name some-geonetwork -d geonetwork
The current docker-entrypoint.sh
file checks the existence of the GeoNetwork DB by a select in pg_database with the line:
if psql -h "$db_host" -U "$POSTGRES_DB_USERNAME" -p "$db_port" -tqc "SELECT 1 FROM pg_database WHERE datname = '$db_name'" | grep -q 1; then
Anyway pgsql
will try and connect to a DB with the same name of the user if a db is not specified.
In the case no DB exists with such a name, the script will misbehave.
We could use the default postgres
DB for the test connection.
Current image has a docker-only param DATA_DIR
which sets geonetwork.dir as a java environment parameter in a deploy script
In stead would be better to use the mechanism available in geonetwork to set the data directory with environment variable by using
docker run -e genoetwork.dir=/mnt/gn
Java environment variable takes preference over system environment variable, so you can not override data_dir by geonetwork.dir (data_dir takes default value if not set)
When I try to run a container using this command as listed in the documentation (https://hub.docker.com/_/geonetwork):
$ docker run --name some-geonetwork -d -p 8080:8080 geonetwork
The container immediately exists. Isn't it supposed to stay up?
I installed 4.2.1.I am not sure if it has been installed successfully. it ends up with the following message
Post "http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.41/containers/aa5698b40b1b5d1ae44fef292442ff6431ea6decff782bf19634fb86b476c923/start": context canceled
when i fire up localhost:8080, it give me the following
I think it is a elastic search problem?
see the following elastic search log below:
2022-12-08 19:45:36 "stacktrace": ["org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:601) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:332) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:636) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:415) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$0(AbstractSearchAsyncAction.java:240) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction$2.doRun(AbstractSearchAsyncAction.java:308) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",
2022-12-08 19:45:36 "at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",
2022-12-08 19:45:36 "at java.lang.Thread.run(Thread.java:832) [?:?]",
2022-12-08 19:45:36 "Caused by: org.elasticsearch.action.NoShardAvailableActionException",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:448) ~[elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:397) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "... 9 more"] }
2022-12-08 19:45:36 {"type": "deprecation", "timestamp": "2022-12-08T19:45:36,981Z", "level": "DEPRECATION", "component": "o.e.d.i.m.RootObjectMapper", "cluster.name": "docker-cluster", "node.name": "3fc68831fa82", "message": "dynamic template [dateType] has invalid content [{\"match\":\"ft_*_d\",\"mapping\":{\"format\":\"\",\"type\":\"double\"}}], attempted to validate it with the following match_mapping_type: [object, string, long, double, boolean, date, binary], caused by [unknown parameter [format] on mapper [__dynamic__dateType] of type [double]]", "cluster.uuid": "LFqwZK0jTWeT3sZ4hLgnVA", "node.id": "4LFPR92tS--O-sE1QgGAiA" }
2022-12-08 19:45:38 {"type": "server", "timestamp": "2022-12-08T19:45:38,564Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "3fc68831fa82", "message": "Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.dashboards_1][0]]]).", "cluster.uuid": "LFqwZK0jTWeT3sZ4hLgnVA", "node.id": "4LFPR92tS--O-sE1QgGAiA" }
2022-12-08 19:46:30 {"type": "deprecation", "timestamp": "2022-12-08T19:46:30,196Z", "level": "DEPRECATION", "component": "o.e.d.c.m.IndexNameExpressionResolver", "cluster.name": "docker-cluster", "node.name": "3fc68831fa82", "message": "this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .kibana_task_manager_1], but in a future major version, direct access to system indices will be prevented by default", "cluster.uuid": "LFqwZK0jTWeT3sZ4hLgnVA", "node.id": "4LFPR92tS--O-sE1QgGAiA" }
any solution? Thanks!
I did a PR against that project, does it look right docker-library/docs#1072
In order to ensure data permanence, i am used to sharing data between containers and docker host. With version 3.10.3, I was sharing in particular /config, and the docker composition was working fine.
but with version 4.0.1 and 4.0.2, I encounter an error when I share /catalog-data ...
Exception message if any is /catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd, Error=Raised exception while starting the application.
Fix the error and restart., Handler=org.fao.geonet.Geonetwork, Exception=OperationAbortedEx : Failed whilst adding the schema information.
Exception message if any is /catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd, Stack=OperationAbortedEx :
Failed whilst adding the schema information. Exception message if any is /catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd
below is an extract from my docker-compose.yml:
geonetwork:
image: geonetwork:4.0.2
...
volumes:
- /home/docker/docker-geonetwork/4.0.2/catalogue-data:/catalogue-data
# - geonetwork:/catalogue-data
The file pre.xsd does exist in the local folder, and it is fully read / write accessible.
$ pwd
/home/docker/docker-geonetwork/4.0.2
$ ls -l catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd
-rw-r--r-- 1 docker users 386 août 17 08:41 catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd
any help would be appreciated.
Ernest.
GN v3.6.0 has been released.
Actions: Prepare the new Docker image.
Does somebody have a docker-compose file to share for geonetwork version 3?
I would like to have it running along with NGINX and POSTGRES. I am only seeing docker files in the repo for geonetwork version 3.
I've been trying to deploy a GeoNetwork (GN) using the docker-compose on an Azure VM with OAuth authentification, and I have difficulty with the redirect URL.
My current hypothesis is that since we use the default GN image container, it comes in a Jetty image that does not support the HTTPS request nor properly handle the HTTP into HTTPS redirect (as you explained); thus, URLs are not correctly handled.
Is there a way to inject configuration to Jetty in the docker image?
If I want to package my own GN image in a Dockerfile, what do I need?
Is there a way to have an image with an HTTPS Jetty directly from this image repository (in the future) for easier production use?
When self-hosting GeoNetwork Docker containers, the Catalog Server Host and Port need to be changed in most cases. These settings can be changed as documented using the Webinterface: https://docs.geonetwork-opensource.org/4.2/administrator-guide/configuring-the-catalog/system-configuration/#catalog-description. It would be convenient to configure e.g. the hosting domain via an environment variable from docker-compose.yml so that a deployment can be specified and documented there instead of requiring manual one-off interaction with the webapp after deployment.
When using the container geonetwork:3.2.1-postgres, I get an error similar to core-geonetwork issue 1887 (java.lang.NoSuchMethodError: org.postgresql.core.BaseConnection.getPreferQueryMode()Lorg/postgresql/jdbc/PreferQueryMode;).
If I remove postgresql-9.4.1211.jar, the container starts properly, afterwards.
As a user I want to use Tomcat instead of Jetty in the Docker container, but could not find related information in https://github.com/geonetwork/docker-geonetwork/tree/main/4.4.1.
Vulnerable JavaScript library
/geonetwork/static/lib3d.js
How to remove the above security vulnerability from geonetwork coming after security scan of accunetix
I'm using docker-geonetwork through Continuous Integration to perform metadata CRUD operation integration tests, using the Geonetwork API. Interacting with the API requires to have geonet:info
retrieved in service responses, disabled by default in the Geonetwork installation. Is there a way I could specify this in docker-geonetwork?
Thanks
Emmanuel
in quite cases people deploy geonetwork db in a schema which is not the default schema of the user, the custom schema needs to be set in spring-config.xml. Unfortunately the database update scripts don't use this param from spring-config, for those scripts you also need to set
db=mydb?currentSchema=example
There's no 4.2.6 tag present in https://hub.docker.com/r/geonetwork/geonetwork/tags
Not a big deal, we always can build it
But geonetwork/gn-cloud-ogc-api-records-service:4.2.6-0 does not exist either on docker hub and this might be more tricky to build
It seems that the docker-entrypoint.sh is missing some logic to configure GN to use a postgresql database, if we compare both scripts here:
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.