Code Monkey home page Code Monkey logo

elasticsearch-pipeline-demo's People

Contributors

c-w avatar fnocera avatar margaretmeehan avatar michaelperel avatar

Stargazers

avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

cicorias

elasticsearch-pipeline-demo's Issues

Elasticsearch image issue with OpenShift

TLDR: Need to solidify which ElasticSearch image to use while running ElasticSearch on OpenShift.

Current Environment

Host: Linux Ubuntu 18.04
Minishift:  Running
Profile:    minishift
OpenShift:  Running (openshift v3.11.0+7f5d53b-195)
DiskUsage:  29% of 19G (Mounted On: /mnt/sda1)
CacheUsage: 1.679 GB (used by oc binary, ISO or cached images)

I begin with the docker image docker.elastic.co/elasticsearch/elasticsearch:7.1.1 and it fails to run in the OpenShift (OS) cluster.

Here are the details:

$ cat es-deployment-1.yaml

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: elasticsearch-conf
  labels:
    app: elasticsearch
data:
  elasticsearch.yml: |
    cluster:
      name: elasticsearch
    node:
      name: ${HOSTNAME}
    network:
      host: 0.0.0.0
    discovery:
      zen.ping.unicast.hosts: elasticsearch-cluster
      zen.minimum_master_nodes: 1
    path:
      data: /elasticsearch/persistent/elasticsearch/data
      logs: /elasticsearch/logs
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: elasticsearch
  annotations:
    image.openshift.io/triggers: |
      [{
      "from": {
        "kind": "ImageStreamTag",
        "name": "elasticsearch:7.1.1"
        },
        "fieldPath": "spec.template.spec.containers[?(@.name==\"elasticsearch\")].image"
      }]
  labels:
    app: elasticsearch
spec:
  podManagementPolicy: OrderedReady
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  serviceName: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        env:
        - name: LOG_LEVEL
          value: info
        image: ' '
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9200
          name: api
          protocol: TCP
        - containerPort: 9300
          name: cluster
          protocol: TCP
        resources:
          limits:
            memory: 512Mi
          requests:
            memory: 512Mi
        volumeMounts:
        - mountPath: /elasticsearch/persistent
          name: elasticsearch-persistent
        - mountPath: /etc/elasticsearch/elasticsearch.yml
          subPath: elasticsearch.yml
          name: elasticsearch-conf
      securityContext: {}
      volumes:
      - name: elasticsearch-persistent
        emptyDir: {}
      - name: elasticsearch-conf
        configMap:
          name: elasticsearch-conf
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  labels:
    app: elasticsearch
spec:
  ports:
  - name: api
    port: 9200
    protocol: TCP
    targetPort: 9200
  selector:
    app: elasticsearch
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch-cluster
  labels:
    app: elasticsearch
spec:
  ports:
  - name: cluster
    port: 9300
    protocol: TCP
    targetPort: 9300
  selector:
    app: elasticsearch
  type: ClusterIP
  clusterIP: None
---
apiVersion: v1
kind: ImageStream
metadata:
  name: elasticsearch
  labels:
    app: elasticsearch
spec:
  lookupPolicy:
    local: false
  tags:
  - name: 7.1.1
    from:
      kind: DockerImage
      name: docker.elastic.co/elasticsearch/elasticsearch:7.1.1

The docker.elastic.co/elasticsearch/elasticsearch:7.1.1 image pulled correctly and container get started (initially).

Events:
  Type     Reason     Age               From                Message
  ----     ------     ----              ----                -------
  Normal   Scheduled  2m                default-scheduler   Successfully assigned myproject/elasticsearch-0 to localhost
  Normal   Pulling    2m                kubelet, localhost  pulling image "docker.elastic.co/elasticsearch/elasticsearch@sha256:1084c64eed7d9318d028361c9aee398afdeb70d1816ce81d590b9450ec542c08"
  Normal   Pulled     49s               kubelet, localhost  Successfully pulled image "docker.elastic.co/elasticsearch/elasticsearch@sha256:1084c64eed7d9318d028361c9aee398afdeb70d1816ce81d590b9450ec542c08"
  Warning  BackOff    22s               kubelet, localhost  Back-off restarting failed container
  Normal   Created    9s (x3 over 49s)  kubelet, localhost  Created container
  Normal   Pulled     9s (x2 over 32s)  kubelet, localhost  Container image "docker.elastic.co/elasticsearch/elasticsearch@sha256:1084c64eed7d9318d028361c9aee398afdeb70d1816ce81d590b9450ec542c08" already present on machine
  Normal   Started    8s (x3 over 48s)  kubelet, localhost  Started container

But later run into issues.

$ oc get po
NAME              READY     STATUS             RESTARTS   AGE
elasticsearch-0   0/1       CrashLoopBackOff   11         34m

$ oc logs  po/elasticsearch-0 
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.

After doing some troubleshooting, I fixed this issue by using a specific image(s) from a RedHat repository.

$ cat es-deployment-2.yaml

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: elasticsearch-conf
  labels:
    app: elasticsearch
data:
  elasticsearch.yml: |
    cluster:
      name: elasticsearch
    node:
      name: ${HOSTNAME}
    network:
      host: 0.0.0.0
    discovery:
      zen.ping.unicast.hosts: elasticsearch-cluster
      zen.minimum_master_nodes: 1
    path:
      data: /elasticsearch/persistent/elasticsearch/data
      logs: /elasticsearch/logs
---
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: elasticsearch
  annotations:
    image.openshift.io/triggers: |
      [{
      "from": {
        "kind": "ImageStreamTag",
        "name": "elasticsearch:5.6.10"
        },
        "fieldPath": "spec.template.spec.containers[?(@.name==\"elasticsearch\")].image"
      }]
  labels:
    app: elasticsearch
spec:
  podManagementPolicy: OrderedReady
  replicas: 1
  selector:
    matchLabels:
      app: elasticsearch
  serviceName: elasticsearch
  template:
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        env:
        - name: LOG_LEVEL
          value: info
        image: ' '
        imagePullPolicy: IfNotPresent
        ports:
        - containerPort: 9200
          name: api
          protocol: TCP
        - containerPort: 9300
          name: cluster
          protocol: TCP
        resources:
          limits:
            memory: 512Mi
          requests:
            memory: 512Mi
        volumeMounts:
        - mountPath: /elasticsearch/persistent
          name: elasticsearch-persistent
        - mountPath: /etc/elasticsearch/elasticsearch.yml
          subPath: elasticsearch.yml
          name: elasticsearch-conf
      securityContext: {}
      volumes:
      - name: elasticsearch-persistent
        emptyDir: {}
      - name: elasticsearch-conf
        configMap:
          name: elasticsearch-conf
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch
  labels:
    app: elasticsearch
spec:
  ports:
  - name: api
    port: 9200
    protocol: TCP
    targetPort: 9200
  selector:
    app: elasticsearch
  sessionAffinity: None
  type: ClusterIP
---
apiVersion: v1
kind: Service
metadata:
  name: elasticsearch-cluster
  labels:
    app: elasticsearch
spec:
  ports:
  - name: cluster
    port: 9300
    protocol: TCP
    targetPort: 9300
  selector:
    app: elasticsearch
  type: ClusterIP
  clusterIP: None
---
apiVersion: v1
kind: ImageStream
metadata:
  name: elasticsearch
  labels:
    app: elasticsearch
spec:
  lookupPolicy:
    local: false
  tags:
  - name: latest
    from:
      kind: DockerImage
      name: registry.centos.org/rhsyseng/elasticsearch:latest
  - name: 6.2.2
    from:
      kind: DockerImage
      name: registry.centos.org/rhsyseng/elasticsearch:6.2.2
  - name: 6.1.2
    from:
      kind: DockerImage
      name: registry.centos.org/rhsyseng/elasticsearch:6.1.2
  - name: 5.6.10
    from:
      kind: DockerImage
      name: registry.centos.org/rhsyseng/elasticsearch:5.6.10

After changing the image lookup to registry.centos.org/rhsyseng/elasticsearch pod came up fine.

$ oc get all
NAME                  READY     STATUS    RESTARTS   AGE
pod/elasticsearch-0   1/1       Running   0          6m

NAME                            TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)    AGE
service/elasticsearch           ClusterIP   172.30.202.99   <none>        9200/TCP   6m
service/elasticsearch-cluster   ClusterIP   None            <none>        9300/TCP   6m

NAME                             DESIRED   CURRENT   AGE
statefulset.apps/elasticsearch   1         1         6m

NAME                                           DOCKER REPO                               TAGS                              UPDATED
imagestream.image.openshift.io/elasticsearch   172.30.1.1:5000/myproject/elasticsearch   latest,5.6.10,6.1.2 + 1 more...   6 minutes ago

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.