greptimeteam / greptimedb-operator Goto Github PK

Design the field to describe the share storage, like S3.

Implement the key process of GreptimeDBCluster controller and bring the mock cluster up.

• implement the basic process of controller and bootstrap the mock cluster;
• add basic e2e test framework;
• add basic deployments;
• add install.sh to bootstrap the whole cluster;

The gtctl command is the devops tool for managing greptimdb cluster:

• gtctl create cluster
• gtctl scale cluster datanode 10
• ...

datanode container CrashLoopBackOff，logs is

ERROR greptime: Fatal error occurs! err.msg=Failed to parse config, source: missing field `metasrv_addrs` for key `meta_client_opts` at line 16 column 1 err.code=InvalidArguments err.source=missing field `metasrv_addrs` for key `meta_client_opts` at line 16 column 1 err.backtrace= 0 <snafu::backtrace_shim::Backtrace as snafu::GenerateImplicitData>::generate::hd7ab405b6b515007

int the https://github.com/GreptimeTeam/greptimedb/blob/develop/config/datanode.example.toml，`metasrv_addr` is update to metasrv_addrs

A new http port 4004 is opened for full prometheus query api compatibility. This port should be exposed as well.

To support AWS/GCP/Azure/AliCloud at first.

Use golangci-lint.

Backgroud

The upstream already supports --store-key-prefix in metasrv. We can provide the StoreKeyPrefix in the Meta field to support creating multiple clusters using the same etcd cluster.

The defaulting.go is too ugly to maintain. We should use mergo to make it short and simple.

The current design of Meta component will use etcd and we need to find out the elegant way to manage etcd cluster. For now, we just deploy our own etcd cluster by creating etcd statefulset which is not managable(adding/removing nodes).

The following is some etcd operations:

1. etcd-operator: the project is already archived
2. etcd-cluster-operator: still maintain
3. etcd-druid: use in Gardener project
4. etcd Helm chart

Currently, datanode is stateful and should use StatefulSet instead of deployment.

Currently, GreptimeDB will use the following configurations:

1. command line options;
2. configuration file(toml format);
3. (option) env;

I think 2 and 3 is now supported(maybe just need some small modifications). To support 1, maybe we need to add the new fields and use configmaps.

The components of GreptimeDB should provide some kind of readiness probe and the operator should use the probe to check service readiness to update status.

Create the RFC style documentations of API design.

For the reason of security, we need to configure mTLS of frontend/datanode/metasrv.

Background

When deploying the greptimedb cluster, it's important to set WalDir and CacheDir:

• The data in WalDir should be maintained in some distributed block storage(like aws EBS);
• The data in CacheDir can be maintained in local disk storage to reach better performance;

GreptimeDBCluster is the Kubernetes CRD that describe the desired state of GreptimeDB cluster. For example, we can describe our own greptimedb cluster as the following YAML:

apiVersion: greptime.cloud/v1alpha1
kind: GreptimeDBCluster
metadata:
  name: basic-gt-cluster
  namespace: default
spec:
  httpServicePort: 8080
  grpcServicePort: 9090
  frontend:
    image: greptime/frontend:v0.1-alpha
    replicas: 1
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  meta:
    image: greptime/meta:v0.1-alpha
    replicas: 3
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"
  datanode:
    image: greptime/datanode:v0.1-alpha
    replicas: 1
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

GreptimeDBCluster should describe the following state:

• describe the component spec(frontend/meta/datanode), including their service and storage spec;
• describe the specified cluster settings, like httpServicePort;
• describe the current status of cluster;

• Add basic workflows(build the project and run the tests);

We should use github.com/GreptimeTeam/greptimedb-operator.

Add dashboard component in GreptimeDBCluster.

Background

In many scenarios, users prefer to use standalone greptimedb. We can design the new CRD to support standalone greptimedb, making the greptimedb management easier.

• Add more userful logging;
• Add Events;
• Add pporf;
• Add metrics;

storageclass.storage.k8s.io "standard" not found

apiVersion: greptime.io/v1alpha1
kind: GreptimeDBCluster
metadata:
  annotations:
    meta.helm.sh/release-name: testdb
    meta.helm.sh/release-namespace: greptime
  creationTimestamp: "2023-03-06T17:51:30Z"
  finalizers:
  - greptimedbcluster.greptime.io/finalizer
  generation: 2
  labels:
    app.kubernetes.io/managed-by: Helm
  name: testdb
  namespace: greptime
  resourceVersion: "2154161"
  uid: ff459b09-ab25-4feb-bcae-a9e8d2bbd69d
spec:
  base:
    main:
      image: docker.io/greptime/greptimedb:0.1.0-alpha-20230220-weekly
      resources:
        limits:
          cpu: 500m
          memory: 128Mi
        requests:
          cpu: 250m
          memory: 64Mi
  datanode:
    replicas: 3
    storage:
      storageClassName: hostpath
      storageRetainPolicy: Retain
      storageSize: 10Gi
  frontend:
    replicas: 1
    service: {}
  grpcServicePort: 4001
  httpServicePort: 4000
  initializer:
    image: docker.io/greptime/greptimedb-initializer:0.1.0-alpha.9
  meta:
    etcdEndpoints:
    - etcd.greptime.svc.cluster.local:2379
    replicas: 1
  mysqlServicePort: 4002
  openTSDBServicePort: 4242
  postgresServicePort: 4003
status:
  clusterPhase: Starting
  datanode:
    readyReplicas: 0
    replicas: 3
  frontend:
    readyReplicas: 0
    replicas: 0
  meta:
    etcdEndpoints:
    - etcd.greptime.svc.cluster.local:2379
    readyReplicas: 1
    replicas: 1

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  creationTimestamp: "2023-03-06T17:37:55Z"
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app.greptime.io/component: testdb-datanode
  name: datanode-testdb-datanode-0
  namespace: greptime
  resourceVersion: "2152548"
  uid: 4ad6c51d-9f5e-4678-9a3f-56fc05233cfc
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: standard
  volumeMode: Filesystem
status:
  phase: Pending