Code Monkey home page Code Monkey logo

patchitup-encrypted's Introduction

patchitup-encrypted
Version Code Coverage Donate

Backup your file to a cloud server using minimum bandwidth.

patchitup-encrypted is a way to keep the cloud up-to-date through incremental patches. In a nutshell, this is a pure-Golang library and a CLI tool for creating a client+server that exchange incremental encrypted and compressed patches to overwrite a remote copy to keep it up-to-date with the client's local file. The files on the server stay encrypted so that only the client ever has access to them.

Why? I wrote this program to reduce the bandwidth usage when backing up SQLite databases to a remote server from Raspberry Pis. I have deployed some software on Raspberry Pis that periodically dumps the database to SQL text. Since Raspberry Pi's can die sometimes, I want to keep their data stored remotely. As the databases can get fairly large, a patch from SQL text will only ever be the changed/new records. patchitup-encrypted allows the client to just send to the cloud only the changed/new records and still maintain the exact copy on the cloud. This can massively reduce bandwidth between the client and the cloud.

Why not git? While git basically does this already, its not terribly easy to setup a git server to support multiple users (though gitolite does a great job of simplifying the process). Also, most of the features of git are not necessary for my use-case.

Note: this is an encrypted version of github.com/schollz/patchitup.

Quickstart

In addition to being a Golang library, the patchitup-encrypted is a server+client. To try it, first install patchitup-encrypted with Go:

$ go install -u -v github.com/schollz/patchitup-encrypted/...

Then start a patchitup-encrypted server:

$ patchitup-encrypted -host
Running at http://0.0.0.0:8002

Then you can patch a file:

$ patchitup-encrypted -s http://localhost:8002 -f SOMEFILE
2018-02-23 08:56:44 [INFO] patched 2.4 kB (62.8%) to remote 'SOMEFILE' for 'yourpublickey'

$ vim SOMEFILE # make some edits

$ patchitup-encrypted -s http://localhost:8002 -f SOMEFILE
2018-02-23 08:57:40 [INFO] patched 408 B (9.9%) to remote 'SOMEFILE' for 'yourpublickey'

The first time you patch will basically just send up the gzipped file. Subsequent edits will just send up the patches. The percentage (e.g. 9.9%) specifies the percentage of the entire file size that is being sent (to get an idea of bandwidth savings). The server also will log bandwidth usage.

Roadmap

I would love PRs.

Some ideas I'd like to add:

  • Built-in security (authentication tokens?)
  • Encryption option (to keep data on server private)
  • Allow -rebuild to output to file

License

MIT

Thanks

Logo designed by www.Vecteezy.com

patchitup-encrypted's People

Contributors

schollz avatar

Stargazers

Russell Daly avatar Krystian Charubin avatar

Watchers

James Cloos avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.