Code Monkey home page Code Monkey logo

drebs's Introduction

DREBS (Disaster Recovery for Elastic Block Store)

About

  • DREBS is a tool for taking periodic snapshots of EBS volumes.
  • DREBS is designed to be run on the EC2 host which the EBS volumes to be snapshoted are attached.
  • DREBS supports configurable retention strategies.
  • DREBS supports configurable pre and post snapshot tasks such as dumping databases prior to snapshot for consistency.

Installation & Setup

  1. Clone the repo or install the gem
  2. Output the example configuration to a file: drebs check_config example_config > your_config.yml
  3. Create an AWS account with authorization limited to create, list, & delete snapshots (Example coming soon)
  4. Add AWS API keys for above account to your_config.yml
  5. configure your_config.yml per your backup requirements
  6. test your configuration: drebs check_config your_config.yml && drebs check_cloud your_config.yml
  7. Add Crontab entry: 0 * * * * drebs execute your_config.yml

Todo

Testing notes

  • shell command: If you do: drebs shell some_config you will end up at a shell with @drebs defined and you will be able to access @drebs.db, @drebs.config, & @drebs.cloud. If you set @drebs.cloud to be an instance of TestCloud from the test suite you should be able to execute various functions without actually hitting AWS and so work from your dev box.

  • Due to the nature of drebs being designed to be run from an ec2 you will need to be on your ec2 instance to test many of the AWS interactions.

  • You should be able to verify data on a snapshot by creating an ebs volume from the snapshot, attaching the volume to your instance and then mounting its file system on some mount point - aws docs on using volumes

Copyright 2014 dojo4

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

drebs's People

Contributors

ahoward avatar dj4 avatar milesmatthias avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

drebs's Issues

Snapshot Management - Enhancement

Sorry, for whatever reason I don't have permission on this repo to call it an enhancement, which is what it is. The DREBS snapshot strategy has really served Inspire Commerce well, however there are aspects of it that make long term retention challenging... primarily thinning history (because Sh$t gets expensive). For example, I have 663 snapshots of our DB since August 2014... FOR 2014. 2015 is another 1100 as of this post. That's a LOT of data. What would be awesome, is to say, "after 5 days, thin records to only the snapshot at midnight (in stead of every hour), and after 60 days, thin records to only what was the record on the last day of the month at midnight. Ok, so this might not be the logic you build, but it would be REALLY useful about now to have something like this. Whatever the strategy is, I'd love to see whatever becomes of drebs address this "thinning" necessity over time that we face... otherwised were left with blunt instruments like dump everything older than x... and that isn't very good for our requirements.

when multiple strategies for the same volume, don't delete unless no other strategy is pointing to that snapshot

hours_between: 1
num_to_keep: 2
mount_point: /dev/sda1
pre_snapshot_tasks: 
post_snapshot_tasks: 
|snap-5bd90897:vol-9c52bcb4,snap-a2f9286e:vol-9c52bcb4|active|1|1|2|||/dev/sda1

---
hours_between: 24
num_to_keep: 4
mount_point: /dev/sda1
|snap-6479aca8:vol-9c52bcb4|active|3|24|4|||/dev/sda1

---
hours_between: 96
num_to_keep: 4
mount_point: /dev/sda1
|snap-6479aca8:vol-9c52bcb4|active|75|96|4|||/dev/sda1

snap-6479aca8 doesn't exist anymore because the first strategy deleted it, since it didn't check to see if any other strategies have a pointer to that snapshot before deleting it in EC2.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.