Code Monkey home page Code Monkey logo

blimp's Introduction

Blimp

Blimp keeps large files out of your git repo by storing them on S3. I wrote Blimp to keep my game development repositories small. I needed something that would run on OS X and Windows with minimal setup. Lastly, using it had to be simple enough for people with limited git experience.

Dependencies

  • ruby
  • git
  • aws-sdk gem
  • aws-sdk-core gem
  • aws s3 account and bucket

Installation

OS X

  • gem install aws-sdk

  • gem install aws-sdk-core --pre

  • git clone https://github.com/piinecone/blimp.git ~/cool/blimp/path/

  • add /cool/blimp/path/bin to your $PATH; e.g.:

    echo 'export PATH=$PATH:/cool/blimp/path/bin' >> ~/.bash_profile

  • export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION environment variables; e.g.:

   echo 'export AWS_ACCESS_KEY_ID=blimpman5000' >> ~/.bash_profile
   echo 'export AWS_SECRET_ACCESS_KEY=letmein' >> ~/.bash_profile
   echo 'export AWS_DEFAULT_REGION=us-east-1' >> ~/.bash_profile
  • ensure which blimp returns something
  • ensure /cool/blimp/path/bin/blimp is executable (chmod +x blimp)

Windows

  • install ruby (http://rubyinstaller.org/) (allow it to add ruby to your $PATH)

  • install git and git bash for windows

  • run git bash

  • gem install aws-sdk

  • gem install aws-sdk-core --pre

  • git clone https://github.com/piinecone/blimp.git ~/cool/blimp/path/

  • add ~/cool/blimp/path/bin to your $PATH; e.g.:

    echo 'export PATH=$PATH:~/cool/blimp/path/bin' >> ~/.bash_profile

  • export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION environment variables; e.g.:

   echo 'export AWS_ACCESS_KEY_ID=blimpman5000' >> ~/.bash_profile
   echo 'export AWS_SECRET_ACCESS_KEY=letmein' >> ~/.bash_profile
   echo 'export AWS_DEFAULT_REGION=us-east-1' >> ~/.bash_profile
  • ensure which blimp returns something
  • ensure ~/cool/blimp/path/bin/blimp is executable (chmod +x blimp)

Usage

From a new repo:

  • cd ~/codebase/with/big/files
  • git init
  • blimp init
  • set the blimp bucket name in .git/config
  • blimp watch *.big_file_extension to .gitignore these files and manage them with blimp
  • ... work / git add / git commit / git push ...
  • blimp status
  • blimp push

In an existing repo:

  • git clone internet://codebase/with/big/files
  • blimp init
  • set the bucket name in .git/config
  • blimp pull to download files from S3 for the current SHA
  • ... work / git add / git commit / git push ...
  • blimp status to see what will be committed (sort of; it's just git status -b -s --ignored)
  • blimp push to upload files matching patterns in .blimp to the specified S3 bucket

How it works

Caveats

Todo

  • Integrate with git
  • Don't add file extensions to .gitignore, instead allow git to version references to the files
  • Skip files listed in .gitignore
  • Do not overwrite files if versions differ; instead, create filename_#{sha}.extension
  • Add a blimp prune command to delete S3 objects from older revisions
  • Only blimp push and blimp pull files that have changed according to git
  • Update blimp status to show only the files that differ from their remote counterparts
  • Attach git history as metadata to S3 objects
  • Create an automated test suite
  • Reduce dependencies

Contributing

blimp's People

Contributors

piinecone avatar

Stargazers

 avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

blimp's Issues

blimp is sensitive to capitlization, .gitignore isn't

I happened to grab a file with a capitalized file extension (.MOV)

.gitignore ignores case, so adding *.mov made the file not known to blimp. But blimp cares about case, so I was briefly in a no man's land where my big file was ignored by git and unknown to blimp.

blimp status lists files that are in global .gitignore

I'm not sure if there is a solution to this, but when you run blimp status, it also lists the status for files that are in my global .gitignore. Maybe globally gitignored files should also be ignored by blimp?

For example, I have .DS_Store files in my global gitignore

$ blimp status
## master...origin/master
M .blimp
M .gitignore
!! .DS_Store
!! DSC_0306.MOV

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.