Source repository for the official CrateDB Docker image
Home Page: https://registry.hub.docker.com/_/crate/
License: Apache License 2.0
If you're reading this on Docker Hub, this image is a nightly build of the official Docker image.
For stable and testing versions of the official Docker image, use the official image.
This project is primarily maintained by Crate.io, but we welcome community contributions!
See the contribution docs for more information.
Looking for more help?
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot]")
When running the tests on macOS, with a recent Docker Desktop 4.12.0 (85629), the MountedDataDirectoryTest croaks. @matriv originally discovered this issue on an M1, I have been able to confirm it also happens on x86.
PATH_TO_IMAGE=. zope-testrunner --color --path . -t MountedDataDirectoryTest
Error in test testRun (tests.itests.MountedDataDirectoryTest)
Traceback (most recent call last):
File "/usr/local/Cellar/[email protected]/3.9.14/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/case.py", line 59, in testPartExecutor
yield
File "/usr/local/Cellar/[email protected]/3.9.14/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/case.py", line 592, in run
self._callTestMethod(testMethod)
File "/usr/local/Cellar/[email protected]/3.9.14/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/case.py", line 550, in _callTestMethod
method()
File "/Users/amo/dev/crate/sources/docker-crate/tests/itests.py", line 127, in inner_fn
self.start(cmd=cmd, ports=ports, env=env, volumes=volumes)
File "/Users/amo/dev/crate/sources/docker-crate/tests/itests.py", line 67, in start
self.cli.start(self.container_id)
File "/Users/amo/dev/crate/sources/docker-crate/.venv/lib/python3.9/site-packages/docker/utils/decorators.py", line 19, in wrapped
return f(self, resource_id, *args, **kwargs)
File "/Users/amo/dev/crate/sources/docker-crate/.venv/lib/python3.9/site-packages/docker/api/container.py", line 1109, in start
self._raise_for_status(res)
File "/Users/amo/dev/crate/sources/docker-crate/.venv/lib/python3.9/site-packages/docker/api/client.py", line 270, in _raise_for_status
raise create_api_error_from_http_exception(e)
File "/Users/amo/dev/crate/sources/docker-crate/.venv/lib/python3.9/site-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
raise cls(e, response=response, explanation=explanation)
docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/containers/00976257b6cac412026367c1cf7a72bc493cb85daf07eb80981f6c1569daf675/start: Internal Server Error ("b'Mounts denied: \nThe path /data is not shared from the host and is not known to Docker.\nYou can configure shared paths from Docker -> Preferences... -> Resources -> File Sharing.\nSee https://docs.docker.com/desktop/mac for more info.'")docker.errors.APIError: 500 Server Error for http+docker://localhost/v1.41/containers/00976257b6cac412026367c1cf7a72bc493cb85daf07eb80981f6c1569daf675/start: Internal Server Error ("b'Mounts denied: \nThe path /data is not shared from the host and is not known to Docker.\nYou can configure shared paths from Docker -> Preferences... -> Resources -> File Sharing.\nSee https://docs.docker.com/desktop/mac for more info.'")
The image does not start, there is the following error:
ERROR: [1] bootstrap checks failed
[1]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] by adding `vm.max_map_count = 262144` to `/etc/sysctl.conf` or invoking `sysctl -w vm.max_map_count=262144`
Any ideas how to solve this?
I read that the following command can be ran before starting the image
grep vm.max_map_count /etc/sysctl.conf vm.max_map_count=262144
Hi,
I am trying to use this image in Marathon along with local persistence volumes on DCOS. Since the data folder for crate is chown-ed to crate user , i am unable to read/write to the persistent storage volume once the container is up.
The error log :
[o.e.b.ElasticsearchUncaughtExceptionHandler] [SYN-Crate-master1] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
at org.elasticsearch.bootstrap.StartupExceptionProxy.<init>(StartupExceptionProxy.java:31) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:142) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:118) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:73) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:122) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.cli.Command.main(Command.java:88) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:87) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:80) ~[crate-app-2.2.4.jar:2.2.4]
Caused by: java.lang.IllegalStateException: Failed to create node environment
at org.elasticsearch.node.Node.<init>(Node.java:270) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.node.CrateNode.<init>(CrateNode.java:62) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:199) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:199) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:282) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:138) ~[crate-app-2.2.4.jar:2.2.4]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /data/nodes
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_151]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_151]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_151]
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_151]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_151]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_151]
at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_151]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:223) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.node.Node.<init>(Node.java:267) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.node.CrateNode.<init>(CrateNode.java:62) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:199) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:199) ~[crate-app-2.2.4.jar:2.2.4]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:282) ~[crate-app-2.2.4.jar:2.2.4]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:138) ~[crate-app-2.2.4.jar:2.2.4]
... 6 more
docker-crate/docker-entrypoint.sh
Line 10 in cc32be3
With crash and openssl, we currently install two packages that are not strictly needed to operate CrateDB inside a container.
For edge environments with limited internet bandwidth, it can be relevant to keep the image size to a minimum.
If neither crash nore openssl is installed, the whole RUN yum install [...] command can be removed, which reduces the image size (uncompressed) from 672 MB to 599 MB (11% saving).
In a first short discussion, it was not clear if any cloud components potentially rely on one of the two packages.
See also #218 for an attempt of removing openssl.
Possible options to consider:
minimal image that doesn't include any components that aren't strictly required for running CrateDBIt is designed for applications that come with their own dependencies bundled (e.g. NodeJS, Python).
Hi, I'm not sure how the entire docker thing works, however can I suggest that the /crate/config folder be available as a mountpoint in docker manifest file?
Status: Downloaded newer image for crate:latest
exec: "-p": executable file not found in $PATHFATA[0232] Error response from daemon: Cannot start container d42b08a9e5800df03564b2da93c107870f511e990b8321fdac429e1bff774755: exec: "-p": executable file not found in $PATH
Unfortunately Red Hat has taken more draconian steps to lock out downstream providers like AlmaLinux. It might be wise to switch to a truly open distribution that will not be impacted.
News:
Hey guys,
as the log4j vulnerability has been fixed yesterday in crate 4.6.6 we would really like to update our kubernetes deployment of crate.
But unfortunately the latest version is not available in docker hub (https://hub.docker.com/_/crate, latest available tag is 4.6.5).
Could you please push the version so we can close open vulnerability in our env? (yes, we've set LOG4J_FORMAT_MSG_NO_LOOKUPS already)
We are using CentOS 7 for our base images. While this distribution still receives security patches, their Docker images apparently stopped being updated. So, we need to invoke yum upgrade at build time, which is officially not allowed in the official Docker images. Their maintainers stated:
Normally I'd say that we don't allow
yum upgradein the official images, but in this case it is becausecentos:7is basically unmaintained since it hasn't been updated since docker-library/official-images#9082.
In December 2020, Red Hat unilaterally terminated CentOS development. In response, CentOS founder Gregory Kurtzer created the Rocky Linux project as a successor to the original mission of CentOS. In March 2021, Cloud Linux (makers of CloudLinux OS) released a new RHEL derivative called AlmaLinux.
While the distribution was discontinued at the end of 2021, development of CentOS Stream, its midstream variant, continues.
We will need to determine a successor distribution. If we want to stay around the same area as before, using a distribution based on packages from Red Hat, I think we can either use:
CentOS Stream is a midstream Linux distribution situated between the upstream development in Fedora and the downstream development for RHEL. The initial release, CentOS Stream 8, was released on 24 September 2019, at the same time as CentOS 8, followed by CentOS Stream 9 on 3 December 2021.
Rocky Linux is a Linux distribution developed by Rocky Enterprise Software Foundation. It is intended to be a downstream, complete binary-compatible release using the Red Hat Enterprise Linux (RHEL) operating system source code. The first release candidate version of Rocky Linux was released on April 30, 2021, and its first general availability version was released on June 21, 2021.
AlmaLinux is a free and open source Linux distribution, created originally by CloudLinux to provide a community-supported, production-grade enterprise operating system that is binary-compatible with Red Hat Enterprise Linux (RHEL). The first stable release of AlmaLinux was published on March 30, 2021.
Using no special settings
docker run -p 4200:4200 -p 4300:4300 -p 5432:5432 crate crate -Cnetwork.host=_site_
I get the following:
[2017-08-21T20:55:45,964][INFO ][o.e.b.BootstrapChecks ] [Ortler] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
ERROR: [1] bootstrap checks failed
[1]: initial heap size [268435456] not equal to maximum heap size [524288000]; this can cause resize pauses and prevents mlockall from locking the entire heap
I am trying to run a django app in docker container.I can run it using cli. But when using api the response is executable not found in $PATH. I have tried changing ENV in api file also.
Hi.
crash stopped working after CentOS move.
You use python36u package which intentionally does not create /usr/bin/python3 symlink.
See iusrepo/python36#2 and iusrepo/packaging#1.
So
ln -sf /usr/bin/python3 /usr/bin/python creates broken link to nowhere#!/usr/bin/env python3 crash shebang rises errorHi,
I am trying to build crate for ARM64v8 architecture and I am getting below error:
/data # crate
[2018-07-13T09:02:39,750][WARN ][o.e.b.Natives ] unable to load JNA native support library, native methods will be disabled.
java.lang.UnsatisfiedLinkError: /tmp/jna-3506402/jna1009797097459244164.tmp: Error loading shared library ld-linux-aarch64.so.1: No such file or directory (needed by /tmp/jna-3506402/jna1009797097459244164.tmp)
at java.lang.ClassLoader$NativeLibrary.load(Native Method) ~[?:1.8.0_171]
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941) ~[?:1.8.0_171]
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824) ~[?:1.8.0_171]
at java.lang.Runtime.load0(Runtime.java:809) ~[?:1.8.0_171]
at java.lang.System.load(System.java:1086) ~[?:1.8.0_171]
at com.sun.jna.Native.loadNativeDispatchLibraryFromClasspath(Native.java:851) ~[jna-4.2.2.jar:4.2.2 (b0)]
at com.sun.jna.Native.loadNativeDispatchLibrary(Native.java:826) ~[jna-4.2.2.jar:4.2.2 (b0)]
at com.sun.jna.Native.<clinit>(Native.java:140) ~[jna-4.2.2.jar:4.2.2 (b0)]
at java.lang.Class.forName0(Native Method) ~[?:1.8.0_171]
at java.lang.Class.forName(Class.java:264) ~[?:1.8.0_171]
at org.elasticsearch.bootstrap.Natives.<clinit>(Natives.java:45) [crate-app-3.0.3.jar:3.0.3]
at org.elasticsearch.bootstrap.BootstrapProxy.initializeNatives(BootstrapProxy.java:104) [crate-app-3.0.3.jar:3.0.3]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:162) [crate-app-3.0.3.jar:3.0.3]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:267) [crate-app-3.0.3.jar:3.0.3]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) [crate-app-3.0.3.jar:3.0.3]
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:135) [crate-app-3.0.3.jar:3.0.3]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:85) [crate-app-3.0.3.jar:3.0.3]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.1.4.jar:3.0.3]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.1.4.jar:3.0.3]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:88) [crate-app-3.0.3.jar:3.0.3]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:81) [crate-app-3.0.3.jar:3.0.3]
[2018-07-13T09:02:39,765][WARN ][o.e.b.Natives ] cannot check if running as root because JNA is not available
[2018-07-13T09:02:39,765][WARN ][o.e.b.Natives ] cannot install system call filter because JNA is not available
[2018-07-13T09:02:39,767][WARN ][o.e.b.Natives ] cannot register console handler because JNA is not available
[2018-07-13T09:02:39,770][WARN ][o.e.b.Natives ] cannot getrlimit RLIMIT_NPROC because JNA is not available
[2018-07-13T09:02:39,770][WARN ][o.e.b.Natives ] cannot getrlimit RLIMIT_AS beacuse JNA is not available
[2018-07-13T09:02:39,770][WARN ][o.e.b.Natives ] cannot getrlimit RLIMIT_FSIZE because JNA is not available
Can you please have a look and let me know what is missing.
If this gets resolved then Crate can be extended for arm64v8 as well
#83 : checked this too.
Regards,
I want to build this image on arm64 architecture, but I'm not sure that can be support arm64 architecture https://cdn.crate.io/downloads/releases/crate-2.1.5.tar.gz
If the health check curl command takes longer than the health check timeout (default is 30 seconds, I believe), then docker will kill the shell running the health check command but not the child processes of that shell, i.e. curl. So in cases where Crate is slow to respond, the health check leaves behind zombie processes that never get reaped. This will eventually fill up the PID table not only in the container but on the docker host machine as well. The simplest solution would probably be to make PID 1 something like tini, which is available in the Alpine Linux package repositories. The end result would be adding/modifying the Dockerfile akin to the following:
# ...
RUN apk add tini
ENTRYPOINT ["/sbin/tini", "--"]
CMD ["/docker-entrypoint.sh", "crate"]I may be able to make a PR for this at a later time if needed.
I was searching for version 3.3.6 (as im facing a memory issue), but it seems the image hasn't been built.
Due to breaking changes from 3.x to 4.x, im restricted version 3.x
Thank you
Starting a cratedb in docker and then connecting via curl gives connection reset. Tried on different servers, same result.
I run container with following command:
docker run --name crate -d --memory 2g --env CRATE_HEAP_SIZE=1g -v /srv/crate:/data -p 4200:4200 -p 4300:4300 crate crate -Ccluster.name=cluster -Clicense.enterprise=false
And get following results when trying to access it per documentation:
curl: (56) Recv failure: Connection reset by peer
I run crate in docker, but I want to know how can I save log into the file? Because I just can see the log in the docker console.
Thank you,
crate:4.5.5 fails to build on arm64v8, but it just looks like the binaries were not published on https://cdn.crate.io/downloads/releases/cratedb/aarch64_linux/.
Dockerfile part being run:
Lines 22 to 23 in 53f8d63
Relevant output:
curl: (22) The requested URL returned error: 404 Not FoundHi, I'm looking to enable docker containers for packages like crate for multiple architectures on dockerhub, starting off with a focus on ppc64le. Wanted to check on the work that will be involved to do that
I have locally been able to successfully test ppc64le changes to the dockerfiles in this repo - changes for enabling additional arch would be minimum.
Would like to know how a multi-arch image for this can be initiated? The official images have this method for enabling multi-arch : https://github.com/docker-library/official-images#multiple-architectures, however this repo isnt an official image yet. Would the first step be to make this an official image or are there any other ways the dockerhub crate image can be enabled for ppc64le?
I'm not deploying on Linux (but a local Triton setup, which uses lx-branded zones), thus there's no Linux kernel, or seccomp and thus the syscall filter is obsolete. I've read in the elasticsearch docs, that the syscall filter can be disabled in elasticsearch.yml.
Is there a way to disable the syscall filter in the docker-crate variant? Going through the components in the Dockerfile, I haven't found an elasticsearch.yml to add this parameter to, in order to rebuild the container and try to deploy it.
-XX:+UnlockExperimentalVMOptions -XX:+UseCGroupMemoryLimitForHeap are also not needed, but I could remove these easily from the docker-entrypoint.sh
But where would I disable the syscall filtering?
When i run docker pull crate/crate:latest i get crate version 0.44.6.
Even if I specify the version number 0.45.2 when doing pull, i get 0.44.6.
Looking at https://registry.hub.docker.com/u/crate/crate/dockerfile/, which I guess is used for latest, it seems like it is not the latest version in docker.
Am I missing something or is there something wrong here?
Hi all,
When trying to import a large CSV file from a HTTPS server, crate complains about certificate validation:
cr> COPY taxis_raw FROM 'https://******' WITH (format='csv') RETURN SUMMARY;
+-----------------------------------------------------+---------------------------------------------------------+---------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| node | uri | success_count | error_count | errors |
+-----------------------------------------------------+---------------------------------------------------------+---------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"id": "fJHPU8JVRxaRgCeMvrgMCg", "name": "Hocharn"} | https://**** | NULL | NULL | {"PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target": {"count": 1, "line_numbers": []}} |
+-----------------------------------------------------+---------------------------------------------------------+---------------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
COPY 1 row in set (0.087 sec)
cr>
Maybe cratedb docker container is missing some CA configuration?
Thanks for your support.
Hi there,
I've discovered a glitch when using crash from the distribution Docker image.
With kind regards,
Andreas.
When running crash from the distribution Docker image, it looks like something can't properly handle non-ASCII characters, like umlauts.
docker run -it --rm --network=host crate:5.1.2 crash -c "SELECT 'äöü';"
+----------+
| '������' |
+----------+
| ������ |
+----------+
docker run -it --rm --network=host crate:5.1.2 crash
cr> SELECT '???';
+-------+
| 'WARNING: Unicode characters found that cannot be displayed. Check your system locale.When running crash on my workstation, everything works flawlessly.
crash -c "SELECT 'äöü';"
CONNECT OK
+-------+
| 'äöü' |
+-------+
| äöü |
+-------+
I have added password based authentication for crate and tried to reuse that image. When this image is reused, passwords are not getting stored. Please help me to fix this issue.
I have used below docker run command
docker run -e CRATE_JAVA_OPTS="-Xms1024m -Xmx1024m" -p 4200:4200 -p 4300:4300 -p 5432:5432 --env CRATE_HEAP_SIZE=2g crate crate -Cnetwork.host=site
it works fine.
I have tried overriding the crate.yml with license and Host Based Authentication Configuration
docker run -v /crate/config/crate.yml:/crate/config/crate.yml -e CRATE_JAVA_OPTS="-Xms1024m -Xmx1024m" -p 4200:4200 -p 4300:4300 -p 5432:5432 --env CRATE_HEAP_SIZE=2g crate crate -Cnetwork.host=site
i get below error any , suggestions or references are appreicated
I have attached crate.yml ( note removed license data )
crate-yaml.zip
Exception in thread "main" ElasticsearchParseException[malformed, expected settings to start with 'object', instead was [VALUE_STRING]]
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:73)
at org.elasticsearch.common.settings.loader.XContentSettingsLoader.load(XContentSettingsLoader.java:52)
at org.elasticsearch.common.settings.loader.YamlSettingsLoader.load(YamlSettingsLoader.java:50)
at org.elasticsearch.common.settings.Settings$Builder.loadFromStream(Settings.java:1037)
at org.elasticsearch.common.settings.Settings$Builder.loadFromPath(Settings.java:1026)
at org.elasticsearch.node.internal.CrateSettingsPreparer.prepareEnvironment(CrateSettingsPreparer.java:67)
at org.elasticsearch.bootstrap.BootstrapProxy.initialEnvironment(BootstrapProxy.java:219)
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:259)
at io.crate.bootstrap.CrateDB.init(CrateDB.java:117)
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:108)
As I understand, it's not a support request, but the official Crate's docker image bug. That's why I've created this issue request.
Here is my Crate's docker-compose.yml setup
general-crate:
image: crate:latest
ports:
- 4200:4200
environment:
- CRATE_HEAP_SIZE=3g
volumes:
- /crate:/data
- <path_to_the_project>/manager/configs/active/services/general-crate/crate.yml:/crate/config/crate.yml
deploy:
mode: global
endpoint_mode: dnsrr
update_config:
parallelism: 2
CrateDB version
3.1.2
Environment description (inside a container)
JVM version: "1.8.0_191
Kernel: Linux ac75a799d53e 4.15.0-42-generic #45-Ubuntu SMP Thu Nov 15 19:32:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Distribution: CentOS-7
Docker: 18.09.0, build 4d60db4
1 node, locally, using docker-compose up (not in swarm mode).
Environment description (where I start a docker container)
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
Description
I try to start a container with official Crate docker image. But it is not starting because of the error described below. It has some access troubles inside the container (AccessDeniedException: /crate/data). If I'm not mistaken, this access issue is an inner issue of the official Crate docker image. Otherwise I don't understand how my docker-compose setup or environment could provoke this?
docker logs stage_general-crate_1_74687e5aa3a0
[2018-12-07T10:29:23,109][INFO ][o.e.n.Node ] [Piz S-chalembert] initializing ...
[2018-12-07T10:29:23,134][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [Piz S-chalembert] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
at org.elasticsearch.bootstrap.StartupExceptionProxy.<init>(StartupExceptionProxy.java:31) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:159) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:135) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:85) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.1.4.jar:3.1.2]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.1.4.jar:3.1.2]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:88) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:81) ~[crate-app-3.1.2.jar:3.1.2]
Caused by: java.lang.IllegalStateException: Failed to create node environment
at org.elasticsearch.node.Node.<init>(Node.java:268) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.node.CrateNode.<init>(CrateNode.java:66) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:267) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) ~[crate-app-3.1.2.jar:3.1.2]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /crate/data
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_181]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_181]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_181]
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_181]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_181]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_181]
at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_181]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:204) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.node.Node.<init>(Node.java:265) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.node.CrateNode.<init>(CrateNode.java:66) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:267) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) ~[crate-app-3.1.2.jar:3.1.2]
... 6 more
Crash does not work in new Crate:3.3.5, ImageID: 89de81728a9d.
Expected:
crash
CONNECT OK
cr>
Running command 'crash' gives an error instead:
/usr/bin/env: python3: No such file or directory
Previously I have seen this work in at least following 3.3.5 image:
crate:sha256:1220d3a66ba7434b40edd63308e7ebb931f7d20dfc03a7f40f24846525d96261
Tested on Ubuntu machines, in Docker and Kubernetes.
Steps to reproduce:
Run container
sudo docker run --name test crate:3.3.5
Exec inside the container
sudo docker exec -it test /bin/bash
Use crash
crash
When I docker pull crate:latest I'm getting 0.52.4. I'd expect at least 0.54.3 since that's the latest released version according to crate.io.
Since crash is not distributed as part of CrateDB release tarball anymore, it will be nice to add crash standalone into Dockerfile.
I have some broken automation scenarios with latest images because of this:
"exec: "crash": executable file not found in $PATH"
I have build the Dockerfile_3.1.j for version 3.2.4 on POWER architecture after applying the required elasticsearch image patch. Dockerfile.txt
However, I am facing runtime issues related to NullPointerException. Following log is observed after executing $docker run -it -p 4200:4200 -p 4300:4300 crate:3.2.4
org.elasticsearch.bootstrap.StartupException: java.lang.NullPointerException
at org.elasticsearch.bootstrap.StartupExceptionProxy.<init>(StartupExceptionProxy.java:31) ~[crate-app-3.2.4.jar:?]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:159) ~[crate-app-3.2.4.jar:?]
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:135) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:85) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.5.1.jar:?]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.5.1.jar:?]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:88) ~[crate-app-3.2.4.jar:?]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:81) ~[crate-app-3.2.4.jar:?]
Caused by: java.lang.NullPointerException
at org.elasticsearch.bootstrap.BootstrapChecks$AllPermissionCheck.isAllPermissionGranted(BootstrapChecks.java:707) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapChecks$AllPermissionCheck.check(BootstrapChecks.java:697) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapChecks.check(BootstrapChecks.java:137) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapChecks.check(BootstrapChecks.java:93) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapChecks.check(BootstrapChecks.java:75) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapProxy$1.validateNodeBeforeAcceptingRequests(BootstrapProxy.java:199) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.node.Node.start(Node.java:755) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapProxy.start(BootstrapProxy.java:206) ~[crate-app-3.2.4.jar:?]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:266) ~[crate-app-3.2.4.jar:?]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) ~[crate-app-3.2.4.jar:?]
Any suggestion would be helpful.
There is a way to send an SQL file on the start of the container
I noticed in the Memory configuration section it says the heap should be set to 1GB minimum.
Should the 512mb limit be updated in the Docker files to reflect this?
Hello all, I discovered what seems a bug in the arm64v8 docker image:
The third step as shown by dive is:
curl --retry 8 -o /openjdk.tar.gz https://download.java.net/java/GA/jdk13.0.1/cec27d702aa74d5a8630c65ae61e4305/9/GPL/openjdk-13.0.1_linux-x64_bin.tar.gz && echo "2e01716546395694d3fad54c9b36d1cd46c5894c06f72d156772efbcf4b41335 */openjdk.tar.gz" | sha256sum -c - && tar -C /opt -zxf /openjdk.tar.gz && rm /openjdk.tar.gz
As you can see is using x86 Java to build the ARM version image.
As a result of this, a simple run of the image: docker run -it -p 4200:4200 crate results in:
/crate/bin/crate: line 129: /opt/jdk-13.0.1/bin/java: cannot execute binary file
/crate/bin/crate: line 129: /opt/jdk-13.0.1/bin/java: Success
What do you thing about this?
Thank you for your time and dedication
As I understand, it's not a support request, but the official Crate's docker image bug. That's why I've created this issue request.
Here is my Crate's docker-compose.yml setup
general-crate:
image: crate:latest
ports:
- 4200:4200
environment:
- CRATE_HEAP_SIZE=3g
volumes:
- /crate:/data
- <path_to_the_project>/manager/configs/active/services/general-crate/crate.yml:/crate/config/crate.yml
deploy:
mode: global
endpoint_mode: dnsrr
update_config:
parallelism: 2
CrateDB version
3.1.2
Environment description (inside a container)
JVM version: "1.8.0_191
Kernel: Linux ac75a799d53e 4.15.0-42-generic #45-Ubuntu SMP Thu Nov 15 19:32:57 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
Distribution: CentOS-7
Docker: 18.09.0, build 4d60db4
1 node, locally, using docker-compose up (not in swarm mode).
Environment description (where I start a docker container)
Distributor ID: Ubuntu
Description: Ubuntu 18.04.1 LTS
Release: 18.04
Codename: bionic
Description
I try to start a container with official Crate docker image. But it is not starting because of the error described below. It has some access troubles inside the container (AccessDeniedException: /crate/data). If I'm not mistaken, this access issue is an inner issue of the official Crate docker image. Otherwise I don't understand how my docker-compose setup or environment could provoke this?
docker logs stage_general-crate_1_74687e5aa3a0
[2018-12-07T10:29:23,109][INFO ][o.e.n.Node ] [Piz S-chalembert] initializing ...
[2018-12-07T10:29:23,134][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [Piz S-chalembert] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalStateException: Failed to create node environment
at org.elasticsearch.bootstrap.StartupExceptionProxy.<init>(StartupExceptionProxy.java:31) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:159) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.execute(CrateDB.java:135) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:85) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) ~[elasticsearch-cli-6.1.4.jar:3.1.2]
at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-cli-6.1.4.jar:3.1.2]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:88) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.main(CrateDB.java:81) ~[crate-app-3.1.2.jar:3.1.2]
Caused by: java.lang.IllegalStateException: Failed to create node environment
at org.elasticsearch.node.Node.<init>(Node.java:268) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.node.CrateNode.<init>(CrateNode.java:66) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:267) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) ~[crate-app-3.1.2.jar:3.1.2]
... 6 more
Caused by: java.nio.file.AccessDeniedException: /crate/data
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84) ~[?:1.8.0_181]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) ~[?:1.8.0_181]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) ~[?:1.8.0_181]
at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:384) ~[?:1.8.0_181]
at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_181]
at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_181]
at java.nio.file.Files.createDirectories(Files.java:767) ~[?:1.8.0_181]
at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:204) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.node.Node.<init>(Node.java:265) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.node.CrateNode.<init>(CrateNode.java:66) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy$1.<init>(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.setup(BootstrapProxy.java:202) ~[crate-app-3.1.2.jar:3.1.2]
at org.elasticsearch.bootstrap.BootstrapProxy.init(BootstrapProxy.java:267) ~[crate-app-3.1.2.jar:3.1.2]
at io.crate.bootstrap.CrateDB.init(CrateDB.java:155) ~[crate-app-3.1.2.jar:3.1.2]
... 6 more
Hello Folks, thank you for the container :)
I can not make version 2.x work with travis-ci 🤔
before_install:
- docker pull crate:1.1.6
- docker run -d -p 4200:4200 -p 4300:4300 crate:1.1.6
before_install:
- docker pull crate
- docker run -d -p 4200:4200 -p 4300:4300 crate crate -Ccluster.name=my_cluster -Cnetwork.host=_site_
Which results in:
(https://travis-ci.org/megastef/node-crate/jobs/269288302)
Thank you for any help!
Only a hint to update the readme, I mean this
https://github.com/crate/docker-crate#use-custom-crate-configuration
Here my example:
cat /tmp/conf/crate.yml
discovery:
zen.minimum_master_nodes: 2
gateway:
recover_after_nodes: 3
recover_after_time: 5m
expected_nodes: 3
docker run --name crate-1 -p 4200:4200 -p 4300:4300 -v /tmp/conf:/conf crate:0.55.2 crate -Des.config=/conf/crate.yml -Des.cluster.name=my-cluster
[2016-08-11 12:31:48,737][INFO ][bootstrap ] es.config is no longer supported. crate.yml must be placed in the config directory and cannot be renamed.
I'm really sad about using something like this
docker run -d -p 4200:4200 -p 4300:4300 \
--name crate1-container \
--volume /mnt/data:/data \
--ulimit nofile=65535 \
--ulimit memlock=9223372036854775807 \
--env CRATE_HEAP_SIZE=8g \
crate/crate:latest \
crate \
-Des.cluster.name=crate-cluster \
-Des.node.name=crate1 \
-Des.transport.publish_port=4300 \
-Des.network.publish_host="$HOST" \
-Des.multicast.enabled=false \
-Des.discovery.zen.ping.unicast.hosts="$HOSTS" \
-Des.discovery.zen.minimum_master_nodes=2
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.