Schedule automatic Cassandra backup about beehive-server HOT 3 OPEN

The main thing blocking this is choosing a reliable storage location. One interesting point is, if we're ready to consider S3 as a possible redundancy location, we automatically get an access controlled, directory-like interface to the data stored there. In other words, we could go ahead and start pointing some people to this for pulling datasets.

from beehive-server.

After thinking this over last night, I don't think this is right thing to have for the real deployment. It was more of a emergency reaction to how beehive1 is currently deployed. Cassandra is setup using a single node, so you don't get any of Cassandra's resilience guarantees during failure...

Cassandra is designed specifically to use replication and eventual consistency between a cluster nodes. In production, you'd have a number of nodes running in a cluster so you can drop a certain number of nodes at anytime and still continue running without data loss. Building on top of that reliability feature is the right way to go, if we're using Cassandra.

from beehive-server.

Turns out clustering works beautifully in the example I tried... It only took 10 minutes to setup a 3 node cluster on my own machine and load it with some test data. Using a keyspace with a replication factor of 2, things worked as expected - any single node could be taken completely offline and I still had access to the entire dataset. Just something to think about for production deployment...

from beehive-server.