Comments (2)
Which data stores do we need? Cassandra? MySQL? Elasticsearch? In particular, do we even need MySQL if the other two can cover all our use cases? You could imagine using Cassandra as our data and configuration warehouse and Elasticsearch providing all the searchability and analytics.
from beehive-server.
How do we organize Cassandra tables? Cassandra is very sensitive to how you choose your partition / primary keys, particularly since there's not really a good concept of joins or building additional indices. This often means you need to design a table for a particular query, even if it means duplicating data.
Here's a concrete example: Suppose we want to support both bulk (daily) data pulls and efficient viewing into the last 72 hours of data from a particular node.
We may keep a table partitioned for each node-id+date, as we are now. In addition, we can create a per-node "rolling window" table of recent data partitioned by node-id in a "time sliceable" way and where entries have a TTL of 72 hours. Then, our loader just inserts a copy of the data into both.
from beehive-server.
Related Issues (20)
- Better error message and status codes from beehive-flask HOT 1
- Understand and prototype running core service in clustered mode
- Design and prototype provisioning of core piece of infrastructure HOT 4
- Ensure all important RabbitMQ exchanges and queues are defined as durable HOT 1
- Prototype static version of beehive dataset interface HOT 3
- Ensure SSL/TLS processes have correct configuration and are clear to use HOT 4
- Backup plan for RabbitMQ definitions HOT 1
- Document and review list of metrics + monitoring data in use
- Review ETL processes for sanitization, robustness and correctness HOT 1
- Schedule static dataset index regeneration HOT 1
- Survey "collaborator" beehive deployments
- Clean up Dockerfile and their locations HOT 1
- Start laying out critical Ansible playbooks for provisioning and managing servers
- Layout good places to start "virtual deployment seams" HOT 3
- Review reverse ssh tunnel process HOT 1
- Simplify container runtime management HOT 2
- Unify data model HOT 1
- Ensure update-node-users is enabled during install
- Ensure messages exchange is created during install
- 405 Method Not Allowed on beehive-registration server HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from beehive-server.