ccs-amsterdam / amcat4docker Goto Github PK

I'm collecting ideas what options should be exposed through docker-compose. Currently, one can set:

• location of elastic (if the user wants to run it at a different location and disable the elastic7 container)
• the port of the web client and api (default port is 80)
• the location of the data on the host machine
• some Elasticsearch option:

environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms4g -Xmx4g"

All options for elastic work (since we're using their image), but I assume maybe some should be set by default for amcat users?

See: https://medium.com/rahasak/setup-lets-encrypt-certificate-with-nginx-certbot-and-docker-b13010a12994

Amcat4 documentation suggests using elastic8, but the docker compose files here still use elastic7.

This also makes the "volumes" parameter confusing, because the directory mapping differs between the versions (see: https://stackoverflow.com/questions/65668188/elastic-search-accessdeniedexception-usr-share-elasticsearch-data-nodes-0-a)

elastic8:

  - /path/to/elastic-data:/usr/share/elasticsearch/data # [local path]:[container path]

elastic7:

  - /path/to/elastic-data:/var/lib/elasticsearch/data # [local path]:[container path]

Related to #2, I've been wondering what the best approach would be to back up, make persistent through updates and migrate the elastic data.

Docker claims that "Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. While bind mounts are dependent on the directory structure and OS of the host machine, volumes are completely managed by Docker" (source)

This sounds good, but this answer notes that volumes are only accessible through containers, which does not seem ideal for the backing up and migrating part. In most scenarios it would be possible to create a container and then use it to docker cp $CONTAINER$:path/to/elastic elastic_backup. The only exception is if docker itself is broken on a system. But then it would be okay to access the location /var/lib/docker/volumes/<volume-name> anyway.

When the Elasticsearch container is restarted, amcat4 will throw ConnectionRefusedError: [Errno 111] Connection refused and Exception: Cannot connect to elasticsearch server ('elastic7:9200',) errors until the amcat4 container is restarted as well.

We probably should have an automated way to resolve this, since it can be confusing.

I found the line sudo chown -R 1000:1000 ~/.elasticsearch/database in a SO answer. It solves the problem that the elastic docker container can't access the linked volume on the host machine. However, I'm not sure yet what the implications of this are (especially from a security point of view). A few observations/thoughts:

• The files don't look useful for anyone who wants to access the data
• I can access and even delete ~/.elasticsearch/database without elevated permissions. The data is then gone, although amcat4 seems to think they still exist (which might be a different issue)
• This approach is probably not working on Windows and MacOS

amcat4_client, amcat4_elastic_host, amcat4_host, amcat4_server_name and possibly the volumes and networks should be anchors instead of being set on several containers. This would make the yaml file easier to read and configuration less error prone.