Code Monkey home page Code Monkey logo

mod_rotation_backup's Introduction

Rotation Backup

Zotonic module to create backups following a grandfather-father-child rotation scheme, to store backups in a directory - for instance an external (mounted) NFS or CIFS backup storage space.

  • Configurable backup scheme, for example "6h 1d 1w 1m 1y".
  • Creates separate backups for database and files, optionally with different backup schemes.
  • Automatically removes expired archives.

Inspired by Tarsnapper, with improvements to interval handling.

The backup scheme

The default values will maintain 7 daily backups, 4 weekly backups, 12 monthly backups, and after that one backup per year.

The backup scheme is defined by time interval ranges (default: "1d 1w 1m 1y" - but this can be set to other time values). The first value defines the frequency of the backups; the other values define how many backups should be kept:

  • The smallest interval defines when new backups should be created: as soon as the most recent backup is older than this value (default: 1 day). For the default setting, one backup will be made every day.
    • You can use both 120 and 2h for 2 hours; 3d for 3 days; 6m for six months; and so on. The minimum interval is 10 (minutes) to reduce the load on the server and to prevent overlapping backup tasks.
  • The interval 1d - 1w means: keep 1 daily backup up to 1 week, so 7 backups of the past week.
  • Backups older than the first interval are handles according to the next interval. The interval 1w - 1m means: keep 1 weekly backup up to 1 month, so 4 backups of the past month.
  • The same goes for the next interval 1m - 1y: keep 1 monthly backup up to 1 year, so 12 backups of the past month.
  • The final interval can be read as "1y until the end of time": keep 1 yearly backup.

Archive creation

This is done automatically.

  • Backup names follow the scheme: identifier-job-date-time. For example: mysite-database-20141231-065959.
  • The backup name does not contain information about the interval it belongs to (f.i. "WEEKLY"); the date in the name is used to infer that information. The date is 'universal time', written as dddddd-tttttt.
  • Jobs for Zotonic backups are: database and files.
  • A new backup is created as soon as the most recent backup is older than the first interval.
  • Backups will not be skipped if you activate the module later in the day: when it detects that the most recent backup is older, a new backup is created.
  • Interval settings can be changed at any time.

Archive expiration

This is done automatically.

  • Only archives with the same identifier are considered; archives created for other sites or using different naming schemes are ignored.
  • The date in the archive name is used to infer expiration dates.
  • Calculation starts at the longest interval value (default 1 year). The archive that is closest to that date (the current date minus the interval) is marked as "to keep". Proximity is calculated with a range of plus/minus half an interval (in the example plus or minus half a year).
    • If older archives exist, we go further back in time (the interval value); this process continues until no older archives are found.
    • Archives older than that first interval value that are not marked as "to keep" are marked as "to expire".
  • Then the second longest interval is used, until all intervals have been processed.
  • The most recent archive is always kept.

Some questions you might have

I am seeing more archives than I was expecting

These are extra items to preserve archives when they gradually migrate from new to old.

Internally, archives are grouped into time buckets. When an item expires from a time bucket, it will move to the next bucket (for instance from "days" to "weeks"). This newer archive ("new" from the older bucket point of view) will be kept as a next generation item, until a next item comes along. This mechanism preserves a fresh flow from new to old.

The backup process is slowing the server

The program that compresses files can be resource intensive, but this can be optimized - see "Configuration: nice" below.

Configuration

To be set in Admin > System > Config.

path

REQUIRED

The backup directory path.

Module Key Default value
mod_rotation_backup path

interval

  • The interval range is set with config key interval for module mod_rotation_backup.
  • If not set, the default value will be used: 1d 1w 1m 1y.
  • Intervals are default set for all jobs, or can be further specified for each job: interval_files and interval_database.

These are the default values in /admin/config:

Module Key Default value
mod_rotation_backup interval 1d 1w 1m 1y
mod_rotation_backup interval_files 1d 1w 1m 1y
mod_rotation_backup interval_database 1d 1w 1m 1y

identifier

The default identifier is the site's host name. You can change that with key identifier:

Module Key Default value
mod_rotation_backup identifier [host name]

nice (Performance)

RECOMMENDED

Creating a tar file from a gigabytes-sized directory can bring the server to a standstill. By configuring a priority program, you can run tar with a lower priority.

For example, ionice is a program that sets the io scheduling class and priority for a program (more background). You can set it as: /usr/bin/ionice -c2 -n5.

Using a low priority will result in slower backups, but the server will no longer be hogged.

Module Key Default value
mod_rotation_backup nice

debug

If you are running Zotonic in debug mode, let the module write debug info to the console when set to true:

Module Key Default value
mod_rotation_backup debug false

Installation

Requirements

  • Zotonic 0.11 or higher
  • A NFS/CIFS backup space
    • For local testing you can use a temporary directory

Install

zotonic modules install mod_rotation_backup

Activate

Activate this module in Admin > System > Modules.

mod_rotation_backup's People

Contributors

arthurclemens avatar

Watchers

 avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.