geonetwork / docker-geonetwork Goto Github PK

now is hardcoded an image for use with postgres, maybe we can use an image where dbtype is a parameter, people can decide to use mysql, sqlserver, oracle or postgres
off course we have a challenge to download dynamically latest db driver (or include all?)

h2 could be the default value for dbtype, in that case we would only have a single docker image to maintain

Since v4.4 the individual environment variables for configuring Elasticsearch are obsolete. e.g. ES_HOST, ES_PORT, ES_PASSWORD etc.

This is a regression for us since we rely on passing secrets to individual environment variables from AWS secrets manager/parameter store using the "value_from" directive. See https://docs.aws.amazon.com/AmazonECS/latest/developerguide/secrets-envvar-ssm-paramstore.html#secrets-envvar-ssm-paramstore-update-container-definition

Since we now have to pass all Geonetwork settings in the one variable GN_CONFIG_PROPERTIES it makes it necessary to store the secret for ES_PASSWORD in plain-text in code, unless we store all the values for GN_CONFIG_PROPERTIES as a secret too. However this becomes annoying as we ideally want to be able to look at the code to see what a particular environment is using.

Is there a good reason why these variables were deprecated?

When I try to run a container using this command as listed in the documentation (https://hub.docker.com/_/geonetwork):

$ docker run --name some-geonetwork -d -p 8080:8080 geonetwork

The container immediately exists. Isn't it supposed to stay up?

There's no 4.2.6 tag present in https://hub.docker.com/r/geonetwork/geonetwork/tags

Not a big deal, we always can build it

But geonetwork/gn-cloud-ogc-api-records-service:4.2.6-0 does not exist either on docker hub and this might be more tricky to build

GN v3.6.0 has been released.

Actions: Prepare the new Docker image.

Currently when you access localhost:port you get some file-not-found error, you have to navigate to /geonetwork to open the application

Instead would be better to install geonetwork at root, by:

• adding a context with geonetwork as root
• rename war to ROOT.war

In case of context GeoNetwork can be deployed in /opt/webapps/geonetwork

Example of tomcat context:

<Context 
  docBase="/opt/webapps/geonetwork" 
  path="" 
  reloadable="false" 
/>

Hello, we are using version 3.2.2, but we can not make geonetwork to upload files bigger than 100MB, after looking around we found a file: /usr/local/tomcat/webapps/geonetwork/WEB-INF/config-spring-geonetwork.xml where a code fragment must be placed attending some forums advices, but nothing happens:

<bean id="multipartResolver"
        class="org.springframework.web.multipart.commons.CommonsMultipartResolver">
    <property name="maxUploadSize" value="1000000000"/>
<!-- set to 1GB -->
</bean>

After copying the new file we have restarted de container several times but the exceptions keeps rising on files greater than 100MB.

What others files must be changed or what docker configuration must be set in way to upload big files?

In order to ensure data permanence, i am used to sharing data between containers and docker host. With version 3.10.3, I was sharing in particular /config, and the docker composition was working fine.

but with version 4.0.1 and 4.0.2, I encounter an error when I share /catalog-data ...

Exception message if any is /catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd, Error=Raised exception while starting the application. 
Fix the error and restart., Handler=org.fao.geonet.Geonetwork, Exception=OperationAbortedEx : Failed whilst adding the schema information. 
Exception message if any is /catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd,    Stack=OperationAbortedEx : 
Failed whilst adding the schema information. Exception message if any is /catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd

below is an extract from my docker-compose.yml:

geonetwork:
  image: geonetwork:4.0.2
...
 volumes:
  - /home/docker/docker-geonetwork/4.0.2/catalogue-data:/catalogue-data
    # - geonetwork:/catalogue-data

The file pre.xsd does exist in the local folder, and it is fully read / write accessible.

$ pwd
/home/docker/docker-geonetwork/4.0.2
$ ls -l catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd
-rw-r--r-- 1 docker users 386 août  17 08:41 catalogue-data/resources/xml/schemas/iso19115-3.2018/schema/standards.iso.org/19135/-2/pre/1.0/pre.xsd

any help would be appreciated.

Ernest.

Using 3.12.8-postgres may have a bug in the Feedback configuration for sending emails, as it fails:

Sending the email to the following server failed : email-smtp.eu-west-1.amazonaws.com:2465. Could not connect to SMTP host: email-smtp.eu-west-1.amazonaws.com, port: 2465, response: -1

it also fails with 25, 587 or 2587, and with 465 or 2465

Any ideas?

Hello,

I'm try to harvest standard THREDDS services but whenever I try to do so, GeoNetwork is throwing a NullPointerException with the following stacktrace:

java.lang.NullPointerException
        at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.processServices(Harvester.java:1481)
        at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvestCatalog(Harvester.java:416)
        at org.fao.geonet.kernel.harvest.harvester.thredds.Harvester.harvest(Harvester.java:284)
        at org.fao.geonet.kernel.harvest.harvester.thredds.ThreddsHarvester.doHarvest(ThreddsHarvester.java:76)
        at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester$HarvestWithIndexProcessor.process(AbstractHarvester.java:647)
        at org.fao.geonet.kernel.harvest.harvester.AbstractHarvester.harvest(AbstractHarvester.java:718)
        at org.fao.geonet.kernel.harvest.harvester.HarvesterJob.execute(HarvesterJob.java:69)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:557)

The Webpage UI just shows:

Harvest error error
[] 

This seems to be happening with any THREDDS service (my own or any publicly available).
Some examples: https://www.ncdc.noaa.gov/wct/data.php, https://tds.marine.rutgers.edu/thredds/met/wrf/catalog.html

My setup is running through docker-compose using the following YML:

version: '3.1'
services:
  geonetwork:
    image: geonetwork:3.12.2
    restart: always
    environment:
      DATA_DIR: /var/lib/geonetwork
      GEONETWORK_DB_NAME: /var/lib/geonetwork/db/gn
    volumes:
      - ./geonetwork_data:/var/lib/geonetwork

Does anyone have an idea of what could be causing this? Other harvesters seem to be working fine (i.e. harvesting other GeoNetworks, OGC SOS, etc...)

When self-hosting GeoNetwork Docker containers, the Catalog Server Host and Port need to be changed in most cases. These settings can be changed as documented using the Webinterface: https://docs.geonetwork-opensource.org/4.2/administrator-guide/configuring-the-catalog/system-configuration/#catalog-description. It would be convenient to configure e.g. the hosting domain via an environment variable from docker-compose.yml so that a deployment can be specified and documented there instead of requiring manual one-off interaction with the webapp after deployment.

There are concerns raised here about the provenance of the code in OpenJDK 8 base images, both in terms of OpenJDK builds and older Debian distributions used as a base:
https://mail.openjdk.java.net/pipermail/jdk8u-dev/2019-May/009330.html

And certainly when we run a container security scan on the geonetwork:3.6.0 docker image we are seeing a lot of CVEs. Switching from a debian stretch to an alpine base image has eliminated most or all in other projects.

https://hub.docker.com/_/tomcat - this has 8.5.41-jre8-alpine, 8.5-jre8-alpine, 8-jre8-alpine, jre8-alpine, 8.5.41-alpine, 8.5-alpine, 8-alpine any of which you could consider as a base for the official geonetwork image and would improve the situation somewhat

I did a PR against that project, does it look right docker-library/docs#1072

From next major update Elastic will be essential component of GN, currently is optional, although offers nice extra capabilities. Would be interesting to add environment variables that connect geonetwork to an external (or orchestrated) elastic instance.

Current image has a docker-only param DATA_DIR which sets geonetwork.dir as a java environment parameter in a deploy script

In stead would be better to use the mechanism available in geonetwork to set the data directory with environment variable by using

docker run -e genoetwork.dir=/mnt/gn

Java environment variable takes preference over system environment variable, so you can not override data_dir by geonetwork.dir (data_dir takes default value if not set)

I've been trying to deploy a GeoNetwork (GN) using the docker-compose on an Azure VM with OAuth authentification, and I have difficulty with the redirect URL.

My current hypothesis is that since we use the default GN image container, it comes in a Jetty image that does not support the HTTPS request nor properly handle the HTTP into HTTPS redirect (as you explained); thus, URLs are not correctly handled.

Is there a way to inject configuration to Jetty in the docker image?

If I want to package my own GN image in a Dockerfile, what do I need?

Is there a way to have an image with an HTTPS Jetty directly from this image repository (in the future) for easier production use?

It seems that the property GEONETWORK_DB_CONNECTION_PROPERTIES is not used when using SQL server as the database. We added "encrypt=true;trustServerCertificate=true" to GEONETWORK_DB_CONNECTION_PROPERTIES but according to the logs it still tries to connect with "trustServerCertificate=false"

2024-04-04T11:41:43,255 WARN  [geonetwork.databasemigration] -   - Migration: Exception running migration for version: 4.4.3-0 subversion: SNAPSHOT. Cannot create PoolableConnectionFactory ("encrypt" property is set to "true" and "trustServerCertificate" property is set to "false" but the driver could not establish a secure connection to SQL Server by using Secure Sockets Layer (SSL) encryption: Error: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target. ClientConnectionId:5726bea6-77a8-4256-ba88-69a1833ef109)

I'm not a Java developer by any means but shouldn't jdbc.connectionProperties also be referenced in sqlserver.xml?

When using the container geonetwork:3.2.1-postgres, I get an error similar to core-geonetwork issue 1887 (java.lang.NoSuchMethodError: org.postgresql.core.BaseConnection.getPreferQueryMode()Lorg/postgresql/jdbc/PreferQueryMode;).
If I remove postgresql-9.4.1211.jar, the container starts properly, afterwards.

Starting the Docker image fails with "Environment variable ES_HOST is mandatory". This variable is not mentioned in the Docker image documentation:

This command will start a debian-based container, running a Tomcat web server, with a geonetwork war deployed on the server:
docker run --name some-geonetwork -d geonetwork

The current docker-entrypoint.sh file checks the existence of the GeoNetwork DB by a select in pg_database with the line:

if psql -h "$db_host" -U "$POSTGRES_DB_USERNAME" -p "$db_port" -tqc "SELECT 1 FROM pg_database WHERE datname = '$db_name'" | grep -q 1; then

Anyway pgsql will try and connect to a DB with the same name of the user if a db is not specified.
In the case no DB exists with such a name, the script will misbehave.
We could use the default postgres DB for the test connection.

There is no schema plugins and I have no idea how to add a new one from https://github.com/metadata101/iso19139.mcp

I installed 4.2.1.I am not sure if it has been installed successfully. it ends up with the following message

Post "http://%2F%2F.%2Fpipe%2Fdocker_engine/v1.41/containers/aa5698b40b1b5d1ae44fef292442ff6431ea6decff782bf19634fb86b476c923/start": context canceled

when i fire up localhost:8080, it give me the following

I think it is a elastic search problem?

see the following elastic search log below:

2022-12-08 19:45:36 "stacktrace": ["org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:601) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:332) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:636) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:415) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.lambda$performPhaseOnShard$0(AbstractSearchAsyncAction.java:240) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction$2.doRun(AbstractSearchAsyncAction.java:308) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:732) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130) [?:?]",
2022-12-08 19:45:36 "at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630) [?:?]",
2022-12-08 19:45:36 "at java.lang.Thread.run(Thread.java:832) [?:?]",
2022-12-08 19:45:36 "Caused by: org.elasticsearch.action.NoShardAvailableActionException",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:448) ~[elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "at org.elasticsearch.action.search.AbstractSearchAsyncAction.onShardFailure(AbstractSearchAsyncAction.java:397) [elasticsearch-7.11.1.jar:7.11.1]",
2022-12-08 19:45:36 "... 9 more"] }
2022-12-08 19:45:36 {"type": "deprecation", "timestamp": "2022-12-08T19:45:36,981Z", "level": "DEPRECATION", "component": "o.e.d.i.m.RootObjectMapper", "cluster.name": "docker-cluster", "node.name": "3fc68831fa82", "message": "dynamic template [dateType] has invalid content [{\"match\":\"ft_*_d\",\"mapping\":{\"format\":\"\",\"type\":\"double\"}}], attempted to validate it with the following match_mapping_type: [object, string, long, double, boolean, date, binary], caused by [unknown parameter [format] on mapper [__dynamic__dateType] of type [double]]", "cluster.uuid": "LFqwZK0jTWeT3sZ4hLgnVA", "node.id": "4LFPR92tS--O-sE1QgGAiA"  }
2022-12-08 19:45:38 {"type": "server", "timestamp": "2022-12-08T19:45:38,564Z", "level": "INFO", "component": "o.e.c.r.a.AllocationService", "cluster.name": "docker-cluster", "node.name": "3fc68831fa82", "message": "Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[.dashboards_1][0]]]).", "cluster.uuid": "LFqwZK0jTWeT3sZ4hLgnVA", "node.id": "4LFPR92tS--O-sE1QgGAiA"  }
2022-12-08 19:46:30 {"type": "deprecation", "timestamp": "2022-12-08T19:46:30,196Z", "level": "DEPRECATION", "component": "o.e.d.c.m.IndexNameExpressionResolver", "cluster.name": "docker-cluster", "node.name": "3fc68831fa82", "message": "this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .kibana_task_manager_1], but in a future major version, direct access to system indices will be prevented by default", "cluster.uuid": "LFqwZK0jTWeT3sZ4hLgnVA", "node.id": "4LFPR92tS--O-sE1QgGAiA"  }

any solution? Thanks!

As a user I want to use Tomcat instead of Jetty in the Docker container, but could not find related information in https://github.com/geonetwork/docker-geonetwork/tree/main/4.4.1.

Hi there,

I'm a bit worried, about how the best way to set another admin password when I create a derived docker container.
A lot of other docker images provide this kind of option to set those via env parameter (postgresql, wildfly, ...), but for the geonetwork I found no proper way to do this.

Is there a guideline / tutorial / pointer on how to do that?

I have try to use the docker geonetwork:postgres with docker compose v3.

In my docker compose I have 2 service: one for my postgres database and one for my geonetwork.
I use depends_on and a command to verifiy that the database is working.

The command that I have use for that is
command: sh -c '/usr/bin/wait-for-it.sh myPostgresDocker:5432 -t 60 -- catalina.sh run'
When I used that command I see that the content of the Entrypoint is never executed, so the configuration for the database is never done. (the file jdbc.properties don't have properties like "jdbc.username")

For information , the script "wait-for-it.sh" is comming from that url "https://github.com/vishnubob/wait-for-it".

I don't know If I need to use another method to validate that my db is available or if a change have to be done to be able to use the docker compose v3 and to validate that a database is available before the configuration of the database.

For now I have change the command like that to solve the problem :
"sh -c '/usr/bin/wait-for-it.sh myPostgresDocker:5432 -t 60 -- /entrypoint.sh catalina.sh run'".

I'm using docker-geonetwork through Continuous Integration to perform metadata CRUD operation integration tests, using the Geonetwork API. Interacting with the API requires to have geonet:info retrieved in service responses, disabled by default in the Geonetwork installation. Is there a way I could specify this in docker-geonetwork?

Thanks
Emmanuel

It seems that the docker-entrypoint.sh is missing some logic to configure GN to use a postgresql database, if we compare both scripts here:

• https://github.com/geonetwork/docker-geonetwork/blob/main/4.0.6/docker-entrypoint.sh vs
• https://github.com/geonetwork/docker-geonetwork/blob/main/4.0.0-alpha.1/docker-entrypoint.sh#L11-L44

Does somebody have a docker-compose file to share for geonetwork version 3?

I would like to have it running along with NGINX and POSTGRES. I am only seeing docker files in the repo for geonetwork version 3.

database configuration is injected in scripts from docker only environment variables in a deploy script

However geonetwork already has a mechanism to inject environment parameters into the application, to prevent extra (contradicting) documentation, i recommend to use the default geonetwork injection mechanism

docker run -e jdbc.database=example...

Vulnerable JavaScript library
/geonetwork/static/lib3d.js

How to remove the above security vulnerability from geonetwork coming after security scan of accunetix

I try to create a docker composition so that geonetwork and geoserver connect to the same Postgres/PostGIS database.
So I use geonetwork:postgres image as suggested in the doc to connect to a postgres docker container.

When starting the containers, I get several SQLException errors, and all of them end with:

Initialization of bean failed; nested exception is java.lang.RuntimeException: 
java.sql.SQLException: Cannot create PoolableConnectionFactory
(The authentication type 10 is not supported. Check that you have configured the pg_hba.conf file to include the client's IP address or subnet, and that it is using an authentication scheme supported by the driver.)

Here is my docker-compose.yml v3.1 example file (without the geoserver container) so that you can reproduce the problem :

version: '3.1'

services:

  geonetwork:
    image: geonetwork:postgres
    restart: always
    ports:
      - 8080:8080
    command: bash -c 'while !</dev/tcp/db/5432; do sleep 1; done; sh /entrypoint.sh catalina.sh run'
    environment:
      DATA_DIR: /var/lib/geonetwork_data
      POSTGRES_DB_HOST: db
      POSTGRES_DB_PORT: 5432
      POSTGRES_DB_NAME: ${POSTGRES_GN_DB}
      POSTGRES_DB_USERNAME: ${POSTGRES_USER}
      POSTGRES_DB_PASSWORD: ${POSTGRES_PASS}
    volumes:
      - geonetwork:/var/lib/geonetwork_data

  db:
    image: kartoza/postgis:13.0
    volumes:
      - geo-db-data:/var/lib/postgresql
    ports:
      - "25434:5432"
    environment:
      - POSTGRES_DB=${POSTGRES_DB}
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASS=${POSTGRES_PASS}
      - POSTGRES_PASSWORD=${POSTGRES_PASS}
      - ALLOW_IP_RANGE=${ALLOW_IP_RANGE}
    restart: on-failure
    healthcheck:
      test: "exit 0"

volumes:
  geonetwork:
  geoserver-data:
  geo-db-data:

Notes:

• Environment variables are set in a .env file, but you can specify your own values.
• The database image is kartoza/postgis:13.0 because it is the one suggested by the geoserver docker documentation.

By the way, if I try to set a database service with a different image (like postgis/postgis), everything works fine. Yet, both images set up a postgres v13.0 database with a postgis v3.0.
I guess this is a matter of postgres configuration, with some differences between the images, but I don't know which ones.

Could you elaborate on what postgres/postgis versions are compatible with the geonetwork docker images, and maybe specify this in the documentation ?

I was build this image on arm64 architecture and build successed .So I want to know we can add it to the https://hub.docker.com/u/arm64v8/ ?

Hi,

I've followed the install documentation to persist data, and wanted to use graphicOverview in my catalog.

DATA_DIR=/var/lib/geonetwork_data -v /host/geonetwork-docker:/var/lib/geonetwork_data

I've copied thumbnail images to - /host/geonetwork-docker/data/Thumbnails/Geo_125k/XXXXX.jpg

And edited the record to the below which I thought would pick up the file but it's not working.

gmd:graphicOverview
gmd:MD_BrowseGraphic
gmd:fileName
gco:CharacterString/var/lib/geonetwork_data/data/Thumbnails/Geo_125k/XXXXX.jpg</gco:CharacterString>
</gmd:fileName>
gmd:fileDescription
gco:CharacterStringThumbnail</gco:CharacterString>
</gmd:fileDescription>
</gmd:MD_BrowseGraphic>
</gmd:graphicOverview>

Am I misunderstanding how to use relative paths between docker and host volumes?

Since gn.h2.db is created in /usr/local/tomcat/webapps it is lost every time the container is destroyed.

A workaround is to mount /usr/local/tomcat/webapps/gh.h2.db as a volume, but the preferred method would be to store the database in the DATA_DIR or in another place specified by an environment variable.

in quite cases people deploy geonetwork db in a schema which is not the default schema of the user, the custom schema needs to be set in spring-config.xml. Unfortunately the database update scripts don't use this param from spring-config, for those scripts you also need to set

db=mydb?currentSchema=example