Code Monkey home page Code Monkey logo

snapsync's Introduction

snapsync

The snapsync system is a backup system that creates hourly snapshots (backups) of a given directory using hard links to files in snapshots created earlier to save storage. New or modified files are included in the snapshot and removed files are deleted from the snapshot.

A cleanup script is included to preserve snapshots via a retention scheme. By default this scheme only keeps the following snapshots:

  1. the oldest one
  2. the first backup in a month
  3. the first backup on every Sunday [SNAPSYNC_DOW] of the last 4 [SNAPSYNC_MAXWEEK] weeks
  4. Every first backup of the last 7 [SNAPSYNC_MAXDAY] days
  5. Every backup in the last 24 [SNAPSYNC_MAXHOUR] hours

Using the environment variables given inside the square brackets in the rules above you can modify this scheme.

All files in the snapshots are added to a local locate database enabling users to find their files (Locate).

Usage

  1. create a snapsync directory for all snapshots, e.g. /snapsync

  2. create a specific directory inside snapsync for each snapsynced directory, e.g. /snapsync/home

  3. add the following lines to the crontab jobs of the root for each directory (in this example /home):

    15 * * * * snapsync.sh /home /snapsync/home >> /snapsync/home/snapsync.log
    45 * * * * cleanup.sh /snapsync/home >> /snapsync/home/snapsync.log

For the exact usage of the script run the scripts snapsync.sh and cleanup.sh with arguments to get more information.

Runtime errors are added to error.log inside the snapsync directory.

A sample logrotate script (logrotate_snapsync) is included which can be saved in /etc/logrotate.d/snapsync.

Locate

After each modification to the snapsync system the locate database (mlocate.db) is updated. This allows users to find their files and/or directory using the following command:

locate --database=/snapsync/home/mlocate.db -b <filename>

Dependencies

  • standard Linux or macOS environment. This system has not been tested with Windows.
  • bash
  • rsync
  • updatedb (part of locate)

Separating source and backup

This system allows for easy separation of the source hardware and snapsync (backup) hardware. As the system will always copy new or modified files via rsync these copies can also be stored on attached external devices like usb-disks or network storages. Note that the snapsync.sh script must be run on the source system and the cleanup.sh script can be run on the snapsync system.

License and Copyright

                    GNU GENERAL PUBLIC LICENSE
                       Version 3, 29 June 2007

 Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
 Everyone is permitted to copy and distribute verbatim copies
 of this license document, but changing it is not allowed.

Please read the LICENSE file for more information.

snapsync's People

Contributors

joffreywallaart avatar ronligt avatar

Watchers

 avatar

Forkers

joffreywallaart

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.