Code Monkey home page Code Monkey logo

overcast's Introduction

Overcast Logo

Overcast is a simple, SSH-based cloud management CLI that was designed to make it easy to spin up and manage clusters of servers in a consistent, scriptable way.

Screenshot

Concepts

  1. Instances are any machine you can SSH into.
  2. Clusters are sets of instances.

Features

Define clusters and instances using the command line or by editing .overcast/clusters.json.

$ overcast cluster create db
$ overcast cluster create app
$ overcast instance import app.01 --cluster app --ip 127.0.0.2 \
  --ssh-port 22222 --ssh-key $HOME/.ssh/id_rsa
$ overcast instance import app.02 --cluster app --ip 127.0.0.3 \
  --ssh-port 22222 --ssh-key $HOME/.ssh/id_rsa

Create, snapshot and destroy instances on DigitalOcean or Linode.

# Create a new Ubuntu 12.04 instance on DigitalOcean:
$ overcast digitalocean create db.01 --cluster db

# Create a new Ubuntu 12.04 instance on Linode:
$ overcast linode create db.02 --cluster db

# Configure both instances in parallel:
$ overcast run db install/core install/redis --parallel
$ overcast expose db 22 6379 --parallel

Run multiple commands or multiple scripts on any or all of your instances at once, over SSH. Commands can be run sequentially or in parallel using the --parallel flag.

$ overcast run db install/core install/redis
$ overcast run all uptime "free-m" "df -h" --parallel

Push and pull files between your local machine and any or all of your instances at once. Dynamically rewrite file paths to include the instance name using {instance} in either source or destination path.

$ overcast push app nginx/myapp.conf /etc/nginx/sites-enabled/myapp.conf
$ overcast pull all /etc/nginx/sites-enabled/myapp.conf nginx/{instance}.myapp.conf

Overcast is a thin wrapper around your native SSH client, and doesn't install or leave anything on the servers you communicate with, so Overcast itself has no real attack surface.

A script library and recipe library are included to make it easy to deploy common software stacks and applications. The library was written for Ubuntu, but could be extended to include other distributions.

Installation (OS X/Linux)

  1. Install Node.js if not already installed.

  2. Install Overcast using npm.

    $ npm -g install overcast
  3. You can now use Overcast from any directory.

    $ overcast help

Uninstallation

Since Overcast is just a wrapper around SSH, there is nothing on your remote machines to uninstall. To uninstall Overcast from your local machine:

# To remove the Overcast package:
$ npm -g remove overcast
# Optionally delete your Overcast SSH keys and configuration files:
$ rm -rf $HOME/.overcast

Configuration

Overcast looks for an .overcast directory in the current directory, or in some parent directory, otherwise falling back to $HOME/.overcast. This allows you to have multiple configurations, and to check your cluster definitions and scripts into a repo like source code.

The command overcast init will create a new configuration in the current directory. The config directory looks like this:

/.overcast
  /files            # Files to be pushed to / pulled from instances
  /keys             # SSH keys, can be your own or auto-generated by overcast
    overcast.key
    overcast.key.pub
  /scripts          # Scripts to be run on instances
  clusters.json     # Cluster/instance definitions (see example.clusters.json)
  variables.json    # API keys, etc (see example.variables.json)

Commands

overcast aliases

  overcast aliases
    Return a list of bash aliases for SSHing to your instances.

    To use, add this to your .bash_profile:
    test -f $HOME/.overcast_aliases && source $HOME/.overcast_aliases

    And then create the .overcast_aliases file:
    overcast aliases > $HOME/.overcast_aliases

overcast cluster

  overcast cluster count [name]
    Return the number of instances in a cluster.

    Example:
    $ overcast cluster count db
    > 0
    $ overcast instance create db.01 --cluster db
    > ...
    $ overcast cluster count db
    > 1

  overcast cluster create [name]
    Creates a new cluster.

    Example:
    $ overcast cluster create db

  overcast cluster rename [name] [new-name]
    Renames a cluster.

    Example:
    $ overcast cluster rename app-cluster app-cluster-renamed

  overcast cluster remove [name]
    Removes a cluster from the index. If the cluster has any instances
    attached to it, they will be moved to the "orphaned" cluster.

    Example:
    $ overcast cluster remove db

overcast completions

  overcast completions
    Return an array of commands, cluster names, and instance names for use
    in bash tab completion.

    To enable tab completion in bash, add this to your .bash_profile:

    _overcast_completions() {
      local cur=${COMP_WORDS[COMP_CWORD]}
      COMPREPLY=($(compgen -W "`overcast completions`" -- "$cur"))
      return 0
    }
    complete -F _overcast_completions overcast

overcast digitalocean

  These functions require the following values set in .overcast/variables.json:
    DIGITALOCEAN_CLIENT_ID
    DIGITALOCEAN_API_KEY

  overcast digitalocean create [name] [options]
    Creates a new instance on DigitalOcean.

    The instance will start out using the auto-generated SSH key found here:
    /path/to/.overcast/keys/overcast.key.pub

    You can specify region, image, and size of the droplet using -id or -slug.
    You can also specify an image or snapshot using --image-name.

      Option               | Default
      --cluster CLUSTER    |
      --ssh-port PORT      | 22
      --region-slug NAME   | nyc2
      --region-id ID       |
      --image-slug NAME    | ubuntu-12-04-x64
      --image-id ID        |
      --image-name NAME    |
      --size-slug NAME     | 512mb
      --size-id ID         |

    Example:
    $ overcast digitalocean create db.01 --cluster db --size-slug 1gb --region-slug sfo1

  overcast digitalocean destroy [instance]
    Destroys a DigitalOcean droplet and removes it from your account.
    Using --force overrides the confirm dialog. This is irreversible.

      Option               | Default
      --force              | false

    Example:
    $ overcast digitalocean destroy app.01

  overcast digitalocean droplets
    List all DigitalOcean droplets in your account.

  overcast digitalocean images
    List all available DigitalOcean images. Includes snapshots.

  overcast digitalocean poweron [instance]
    Power on a powered off droplet.

  overcast digitalocean reboot [instance]
    Reboots a DigitalOcean droplet. According to the API docs, "this is the
    preferred method to use if a server is not responding."

    Example:
    $ overcast digitalocean reboot app.01

  overcast digitalocean rebuild [instance] [options]
    Rebuild a DigitalOcean droplet using a specified image name, slug or ID.
    According to the API docs, "This is useful if you want to start again but
    retain the same IP address for your droplet."

      Option               | Default
      --image-slug SLUG    | ubuntu-12-04-x64
      --image-name NAME    |
      --image-id ID        |

    Example:
    $ overcast digitalocean rebuild app.01 --name my.app.snapshot

  overcast digitalocean regions
    List available DigitalOcean regions (nyc2, sfo1, etc).

  overcast digitalocean resize [name] [options]
    Shutdown, resize, and reboot a DigitalOcean droplet.
    If --skipboot flag is used, the droplet will stay in a powered-off state.

      Option               | Default
      --size-slug NAME     |
      --size-id ID         |
      --skipBoot           | false

    Example:
    $ overcast digitalocean resize db.01 --size-slug 2gb

  overcast digitalocean sizes
    List available DigitalOcean sizes (512mb, 1gb, etc).

  overcast digitalocean shutdown [instance]
    Shut down a DigitalOcean droplet.

    Example:
    $ overcast digitalocean shutdown app.01

  overcast digitalocean snapshot [instance] [snapshot-name]
    Creates a named snapshot of a droplet. This process will reboot the instance.

    Example:
    $ overcast digitalocean snapshot app.01

  overcast digitalocean snapshots
    Lists available snapshots in your DigitalOcean account.

overcast expose

  overcast expose [instance|cluster|all] [port...]
    Reset the exposed ports on the instance or cluster using iptables.
    This will fail if you don't include the current SSH port.
    Specifying --whitelist will restrict all ports to the specified address(es).
    These can be individual IPs or CIDR ranges, such as "192.168.0.0/24".

    Expects an Ubuntu server, untested on other distributions.

      Option
      --user=NAME
      --whitelist "IP|RANGE..."
      --whitelist-PORT "IP|RANGE..."

    Examples:
    # Allow SSH, HTTP and HTTPS connections from anywhere:
    $ overcast expose app 22 80 443
    # Allow SSH from anywhere, only allow Redis connections from 1.2.3.4:
    $ overcast expose redis 22 6379 --whitelist-6379 "1.2.3.4"
    # Only allow SSH and MySQL connections from 1.2.3.4 or from 5.6.7.xxx:
    $ overcast expose mysql 22 3306 --whitelist "1.2.3.4 5.6.7.0/24"

overcast exposed

  overcast exposed [instance|cluster|all]
    List the exposed ports on the instance or cluster.
    Expects an Ubuntu server, untested on other distributions.

      Option        | Default
      --user NAME   |

overcast health

  overcast health [instance|cluster|all]
    Export common health statistics in JSON format.
    Expects an Ubuntu server, untested on other distributions.

    Example JSON:
    {
      "my_instance_name": {
        "cpu_1min": 0.53,
        "cpu_5min": 0.05,
        "cpu_15min": 0.10,
        "disk_total": 19592,     // in MB
        "disk_used": 13445,      // in MB
        "disk_free": 5339,       // in MB
        "mem_total": 1000,       // in MB
        "mem_used": 904,         // in MB
        "mem_free": 96,          // in MB
        "cache_used": 589,       // in MB
        "cache_free": 410,       // in MB
        "swap_total": 255,       // in MB
        "swap_used": 124,        // in MB
        "swap_free": 131,        // in MB
        "tcp": 152,              // open TCP connections
        "rx_bytes": 196396703,   // total bytes received
        "tx_bytes": 47183785,    // total bytes transmitted
        "io_reads": 1871210,     // total bytes read
        "io_writes": 6446448,    // total bytes written
        "processes": [
          {
            "user": "root",
            "pid": 1,
            "cpu%": 0,
            "mem%": 0,
            "time": "0:01",
            "command": "/sbin/init"
          }
        ]
      }
    }

overcast help

  Overcast v0.2.9

  Code repo, issues, pull requests:
    https://github.com/andrewchilds/overcast

  Usage:
    overcast [command] [options...]

  Help:
    overcast help
    overcast help [command]
    overcast [command] help

  Commands:
    overcast aliases
    overcast cluster list
    overcast cluster count [name]
    overcast cluster create [name]
    overcast cluster rename [name] [new-name]
    overcast cluster remove [name]
    overcast completions
    overcast digitalocean create [instance] [options]
    overcast digitalocean destroy [instance]
    overcast digitalocean droplets
    overcast digitalocean images
    overcast digitalocean poweron [instance]
    overcast digitalocean reboot [instance]
    overcast digitalocean rebuild [instance] [options]
    overcast digitalocean regions
    overcast digitalocean resize
    overcast digitalocean sizes
    overcast digitalocean shutdown [instance]
    overcast digitalocean snapshot [instance] [snapshot-name]
    overcast digitalocean snapshots
    overcast expose [instance|cluster|all] [port...] [options]
    overcast exposed [instance|cluster|all]
    overcast health [instance|cluster|all]
    overcast info
    overcast init
    overcast instance get [name] [attr...]
    overcast instance import [name] [options]
    overcast instance list [cluster...]
    overcast instance remove [name]
    overcast instance update [name] [options]
    overcast linode boot [instance]
    overcast linode create [instance] [options]
    overcast linode datacenters
    overcast linode destroy [instance]
    overcast linode distributions
    overcast linode kernels
    overcast linode linodes
    overcast linode plans
    overcast linode reboot [instance]
    overcast linode shutdown [instance]
    overcast list
    overcast ping [instance|cluster|all]
    overcast port [instance|cluster|all] [port]
    overcast pull [instance|cluster|all] [source] [dest]
    overcast push [instance|cluster|all] [source] [dest]
    overcast reboot [instance|cluster|all]
    overcast run [instance|cluster|all] [command...]
    overcast run [instance|cluster|all] [file...]
    overcast ssh [instance]

  Config directory:
    /path/to/.overcast

overcast info

  overcast info
    Pretty-prints the complete clusters.json file, stored here:
    /path/to/.overcast/clusters.json

overcast init

  overcast init
    Create an .overcast config directory in the current working directory.
    No action taken if one already exists.

overcast instance

  overcast instance get [name] [attr...]
    Returns the instance attribute(s), one per line.

    Examples:
    $ overcast instance get app.01 ssh-port ip
    > 22
    > 127.0.0.1
    $ overcast instance get app.01 user
    > appuser

  overcast instance import [name] [options]
    Imports an existing instance to a cluster.

      Option               | Default
      --cluster CLUSTER    |
      --ip IP              |
      --ssh-port PORT      | 22
      --ssh-key PATH       | .overcast/keys/overcast.key
      --user USERNAME      | root

    Example:
    $ overcast instance import app.01 --cluster app --ip 127.0.0.1 \
        --ssh-port 22222 --ssh-key $HOME/.ssh/id_rsa

  overcast instance list [cluster...]
    Returns all instance names, one per line. Optionally limit to one or more clusters.

    Examples:
    $ overcast instance list
    $ overcast instance list app-cluster db-cluster

  overcast instance remove [name]
    Removes an instance from the index.
    The server itself is not affected by this action.

    Example:
    $ overcast instance remove app.01

  overcast instance update [name] [options]
    Update any instance property. Specifying --cluster will move the instance to
    that cluster. Specifying --name will rename the instance.

      Option               | Default
      --name NAME          |
      --cluster CLUSTER    |
      --ip IP              |
      --ssh-port PORT      |
      --ssh-key PATH       |
      --user USERNAME      |

    Example:
    $ overcast instance update app.01 --user differentuser --ssh-key /path/to/another/key

overcast linode

  These functions require LINODE_API_KEY property to be set in .overcast/variables.json.
  API keys can be found at https://manager.linode.com/profile/api

  overcast linode boot [instance]
    Boot a powered off linode.

  overcast linode create [name] [options]
    Creates a new Linode.

      Option                    | Default
      --cluster CLUSTER         |
      --datacenter-slug NAME    | newark
      --datacenter-id ID        |
      --distribution-slug NAME  | ubuntu-12-04-lts
      --distribution-id ID      |
      --kernel-id ID            |
      --kernel-name NAME        | Latest 64 bit
      --payment-term ID         | 1 (monthly, if not metered)
      --plan-id ID              |
      --plan-slug NAME          | 2048
      --password PASSWORD       | autogenerated
      --ssh-key KEY_PATH        | /path/to/.overcast/keys/overcast.key
      --ssh-pub-key KEY_PATH    | /path/to/.overcast/keys/overcast.key.pub

    Example:
    $ overcast linode create db.01 --cluster db --datacenter-slug london

  overcast linode datacenters
    List available Linode datacenters.

  overcast linode destroy [instance]
    Destroys a linode and removes it from your account.
    Using --force overrides the confirm dialog. This is irreversible.

      Option                    | Default
      --force                   | false

  overcast linode distributions
    List available Linode distributions.

  overcast linode kernels
    List available Linode kernels.

  overcast linode linodes
    List all linodes in your account.

  overcast linode plans
    List available Linode plans.

  overcast linode resize [instance] [options]
    Resizes a linode to the specified plan. This will immediately shutdown and migrate your linode.

      Option                    | Default
      --plan-id ID              |
      --plan-slug NAME          |

  overcast linode reboot [instance]
    Reboots a linode.

  overcast linode shutdown [instance]
    Shut down a linode.

overcast list

  overcast list
    Short list of your cluster and instance definitions, stored here:
    /path/to/.overcast/clusters.json

overcast ping

  overcast ping [instance|cluster|all]
    Ping an instance or cluster.

      Option    | Default
      --count N | 3

    Examples:
    $ overcast ping app.01
    $ overcast ping db --count 5

overcast port

  overcast port [instance|cluster|all] [port]
    Change the SSH port for an instance or a cluster.
    This command will fail if the new port is not opened by iptables.

    Examples:
    $ overcast port app.01 22222
    $ overcast port db 22222

overcast pull

  overcast pull [instance|cluster|all] [source] [dest]
    Pull a file or directory from an instance or cluster using scp. Source is absolute.
    Destination can be absolute or relative to the .overcast/files directory.

    Any reference to {instance} in the destination will be replaced with the instance name.

      Option
      --user NAME

    Example:
    Assuming instances "app.01" and "app.02", this will expand to:
      - .overcast/files/nginx/app.01.myapp.conf
      - .overcast/files/nginx/app.02.myapp.conf
    $ overcast pull app /etc/nginx/sites-enabled/myapp.conf nginx/{instance}.myapp.conf

overcast push

  overcast push [instance|cluster|all] [source] [dest]
    Push a file or directory to an instance or cluster using scp. Source can be
    absolute, or relative to the .overcast/files directory. Destination is absolute.

    Any reference to {instance} in the source will be replaced with the instance name.

      Option
      --user NAME

    Example:
    Assuming instances "app.01" and "app.02", this will expand to:
      - .overcast/files/nginx/app.01.myapp.conf
      - .overcast/files/nginx/app.02.myapp.conf
    $ overcast push app nginx/{instance}.myapp.conf /etc/nginx/sites-enabled/myapp.conf

overcast reboot

  overcast reboot [instance|cluster|all]
    Reboot an instance or cluster.
    If the instance was created using DigitalOcean or Linode, this will use the provider API,
    otherwise will execute "reboot" command on the server and wait for 60 seconds.

overcast run

  overcast run [instance|cluster|all] [command...]
    Runs a command or series of commands on an instance or cluster.
    Commands will run sequentially unless you use the --parallel flag,
    in which case each command will run on all instances simultanously.

      Option                          | Default
      --env "KEY=VAL KEY='1 2 3'"     |
      --user NAME                     |
      --ssh-key PATH                  |
      --parallel -p                   | false
      --continueOnError               | false

    Examples:
    $ overcast run app --env "foo='bar bar' testing=123" env
    $ overcast run all uptime "free -m" "df -h"

  overcast run [instance|cluster|all] [file...]
    Executes a script file or files on an instance or cluster.
    Script files can be either absolute or relative path.
    Script files will run sequentially unless you use the --parallel flag,
    in which case each file will execute on all instances simultanously.

      Option                          | Default
      --env "KEY=VAL KEY='1 2 3'"     |
      --user NAME                     |
      --ssh-key PATH                  |
      --shell-command "COMMAND"       | bash -s
      --parallel -p                   | false
      --continueOnError               | false

    Relative paths are relative to the cwd, or to these directories:
    /path/to/.overcast/scripts
    /path/to/installed/overcast/scripts

    Example:
    $ overcast run db install/core install/redis

overcast ssh

  overcast ssh [instance]
    Opens an SSH connection to an instance.

    Option
    --ssh-key PATH
    --user NAME

Design Goals & Motivation

There are many server management frameworks out there already, but they generally involve a complex server-client implementation, a steep learning curve, or a giant, monolithic conceptual framework that requires taking a course to understand.

I wanted something that had little to no learning curve, that did only what you asked it to do on the remote machines and nothing more, that just focused on multi-server provisioning and communication and leaves problems like process/state management and system monitoring to tools designed specifically for those problems.

Example App Deployments

Running the Tests

Build Status

git clone https://github.com/andrewchilds/overcast.git
cd overcast
npm test

Upgrading Overcast

npm -g update overcast

Configuration files are left alone during an upgrade.

Contributing

Contributions are welcome. If you've got an idea for a feature or found a bug, please open an issue. If you're a developer and want to help improve Overcast, open a pull request with your changes.

Roadmap

  • Tagging
  • Events
  • AWS EC2 support
  • More comprehensive script/recipe library
  • More test coverage

License

MIT. Copyright © 2014 Andrew Childs.

overcast's People

Contributors

andrewchilds avatar kschrader avatar

Watchers

James Cloos avatar Narava avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.