The Clymene is a time-series data and logs collection platform for distributed systems inspired by Prometheus and Jaeger. Time-series data and logs from various environments can be collected and stored in different types of databases. It can be configured in a variety of architectures. Choose the storage that users can use easily and build a monitoring system with dashboards that users can use easily. Also, Clymene Agent's time series data collection uses fewer resources than Prometheus' remote_write. Various time series data analysis functions will be added.
The Clymene project is a platform for collecting time series data and log data. So there are two pipelines: timeseries and logs. All agents can collect data and store it directly in the database, Or, depending on the situation, you can use the Clymene components to configure a variety of architectures. It can be configured with an architecture that can be used with a lot of traffic. The Clymene Project provides various storage types, so choose a storage type you are familiar with. And build great monitoring systems using familiar dashboards.
Clymene Agent(Getting Started)
dockerhub : bourbonkk/clymene-agent:v2.x.x
redhatQuay: quay.io/clymene/clymene-agent:v2.x.x
The Clymene-agent is service that collects time series data(does not use disks)
- Service Discovery
- Prometheus's Service Discovery feature finds Metric collection endpoints.
- scrape time series data
- Time-series data transfer to gateway(gRPC) (Optional)
- Time-series data transfer to kafka (Optional)
- Time-series data insert to Database(supported DB) (Optional)
Clymene Ingester(Optional) (Getting Started)
dockerhub : bourbonkk/clymene-ingester:v2.x.x
redhatQuay: quay.io/clymene/clymene-ingester:v2.x.x
The Clymene ingester is an optional service responsible for insert time series data loaded on kafka into the database.
- Kafka message consume
- Time-series data insert to Database(supported DB) (Optional)
Clymene Gateway(Optional) (Getting Started)
dockerhub : bourbonkk/clymene-gateway:v2.x.x
redhatQuay: quay.io/clymene/clymene-gateway:v2.x.x
The Clymene Gateway is an optional service that can receive metric data from the another component through gRPC or HTTP
communication.
- gRPC, HTTP Service
- Time-series data insert to Database(supported DB) (Optional)
Clymene Promtail(Getting Started)
dockerhub : bourbonkk/clymene-promtail:v2.x.x
redhatQuay: quay.io/clymene/clymene-promtail:v2.x.x
The Clymene-promtail customized loki's log collection agent for the Clymene project.
- Service Discovery
- log collection
- log data transfer to gateway(gRPC or HTTP)
- log data transfer to kafka
- log data insert to Database(supported DB) (Optional)
Promtail Ingester(Optional) (Getting Started)
dockerhub : bourbonkk/promtail-ingester:v2.x.x
redhatQuay: quay.io/clymene/promtail-ingester:v2.x.x
Promtail ingester is an optional service responsible for insert log data loaded on kafka into the database.
- Kafka message consume
- Time-series data insert to Database(supported DB) (Optional)
Promtail Gateway(Optional) (Getting Started)
dockerhub : bourbonkk/promtail-gateway:v2.x.x
redhatQuay: quay.io/clymene/promtail-gateway:v2.x.x
The Promtail Gateway is an optional service that can receive log data from the another component through gRPC or HTTP
communication.
- gRPC, HTTP Service
- Time-series data insert to Database(supported DB) (Optional)
For example, see the clymene_using_gateway.yaml
For example, see the docker-compose-clymene_using_gateway.yml
- AI/ML platform for clymene(https://github.com/Clymene-project/clymene-analyzer)
- push-type agent(node-exporter, cadvisor, process-exporter, etc)
- Add new pipeline(trace or Something - discussions)
- eBPF trace agent(for v3.0.0 release)
Have questions, suggestions, bug reports? Reach the project community via these channels:
- GitHub issues and discussions
To learn more about Clymene, consult the following talks and articles.
- Jan 2022 blog post "Efficient time series data collection and management plan in a distributed environment"
- Aug 2022 blog post "Best Practices, Monitoring System for Multi-K8s Cluster Environments Using Open Source"
- Aug 2022 blog post "Best practice, k8s Node/POD resource usage and log monitoring system for multi-k8s cluster environment using Open source"
Apache License 2.0, see LICENSE.
The Clymene project is being developed with the support of Open Source development license of Jetbrains'.