Code Monkey home page Code Monkey logo

pantry's Introduction

Pantry: Modern DevOps Automation

Build Status Code Climate

Pantry takes the tedium out of setting up a DevOps stack by providing framework for storing, sharing, and running server provisioning and configuration. Whether your stack is Chef or Puppet, Docker or Packer, or any mix of tools, Pantry doesn't care!

Installation

Install Pantry via Rubygems on all servers and local machines:

gem install pantry

Requirements

Pantry depends on Celluloid and ZeroMQ. Pantry is built for Ruby 2.0 and later and requires Rubygems 2.1 and later.

Usage

Pantry has three main aspects: the Server, a Client, and the CLI. Pantry provides command-line tools for all of these.

Starting the Server: pantry-server -c /path/to/server.yml

Starting a Client: pantry-client -c /path/to/client.yml

Running the CLI: pantry --help

For more information, see Getting Started.

Documentation

The Documentation for Pantry is available at http://pantryops.org and the RDoc is served up at rdoc.info/pantry.

Available Plugins

Project Details

Contributing

  • Fork this repository on Github
  • Make your changes and send us a Pull Request
  • All Pull Requests must contain tests
  • Pull Request tests must pass before being accepted

License

Pantry is distributed under the MIT License. See LICENSE for more details.

pantry's People

Contributors

jasonroelofs avatar

Stargazers

Tom Hensel avatar Ricardo Pontes avatar Frank Müller avatar Jay Perry avatar

Watchers

Daniel Morrison avatar  avatar Chris Gaffney avatar Eric Milford avatar Tim Bugai avatar Jason Carpenter avatar James Cloos avatar Spencer Toth avatar

pantry's Issues

Benchmark memory usage

Make sure that memory stays under control at all times. Potential places of memory explosion:

  • Lots of requests that build up large wait-lists
  • Lots of file uploads

Especially test when situations aren't normal, like a file request never finishes or a request never gets a response.

Invalidate / Remove unused Curve client public keys

For now I'm just adding new client public keys to the server's list of known clients without any real checking. We will definitely want to keep track of which keys are used, which ones aren't used, and clean up the empty ones.

Running Server / Client as Root?

This question has been mulling around in my head for a long time and I'm not sure what the right answer is. I think the Server can be safely run as a non-root user as it shouldn't ever execute anything outside of it's Pantry.root. The Client though is another question. If it's a pantry-chef client, then we need root access to run Chef on the box. This requires either sudoers (and NOPASSWD or a specific config for chef-solo) or just running as root.

The Client only ever connects out to a Server, it never allows connections in. Thus, the only way that one can convince a Client to do something is through a Server. The use of Curve encryption and strong client public key authentication makes it, hopefully, impossible to MITM this setup.

Given that tools like chef-client and puppet-client do need root access, I think we'll be fine to have the Client run as root by default, but wouldn't mind other viewpoints and discussion.

Server: Should run as non-root
Client: Should run as root

Streaming responses of long-running processes

It would be nice to see on-going process of long running processes so as not to feel that the CLI locked up or is waiting for something that's not coming. For example, chef:run, that could stream back stdout as it happens back to the CLI to be displayed to the user.

The main API idea I have right now is to return a proc from #perform and periodically query that proc until it's done. No idea what it would look like or work yet though. This could also then be used to set up remote tailing of logs / files as well.

Client Heartbeat

Every so often the Client should sent another RegisterClient heartbeat message to ensure the Server has a record of it's existence. Should the server ever reboot, this ensures automatic eventual consistency of the network.

"Omnibus" Installer

As pantry requires Ruby 2.0, there's a good chance it will be run on systems that aren't running 2.0. Set up an omnibus installer script like Chef to install a packaged Ruby (maybe for starters just a source compile).

Investigate Serfdom networking layer

Keep an eye on Serfdom and as that enters Production Ready state, investigate if it's feasible to use for networking instead of ZeroMQ.

Main question: encryption and security of communication.

CLI: Allow extra options after arguments?

The current implementation of the command line has strict ordering, so:

chef:cookbook:upload -f path/to/cookbook

is valid where as

chef:cookbook:upload path/to/cookbook -f

is not. If this becomes a regular problem for me or others I'll look into making the option parsing a bit more fluid.

Clean up commands. Remove .from_message?

If Command#perform takes the Message triggering this Command, is .from_message really necessary anymore? Starting to feel like a redundancy at best, confusing at worst.

Security and Authorization improvements

To start, when using the ZMQ Curve crypto layer, Pantry will accept any client who has the Server's public key, and will store locally the client's public key on connect. This should be updated to having the Server generate all keys, and only allowing Clients to connect who were given a key generated by the given Server. Then, if the Server public key does leak, bad clients still cannot actually connect and use the Pantry server.

Chef Provisioner

Time for Pantry to start storing, providing for, and running Chef.

Major pieces that the Server needs to store

Chef-specific pieces:

  • Cookbooks
  • Data Bags

Application specific:

  • Environments
  • Roles
  • Cookbooks (Application's own cookbook and site-cookbook versions of shared cookbooks)

Client must be able to:

  • Ask server for the files it should receive
  • Store received files and configure Chef Solo
  • Run chef solo
  • Report back results to Server

CLI will need to support:

  • Uploading cookbooks, environments, roles, data bags
  • Telling clients to run chef

Pick a command pattern and stick with it

Trying to decide on a good command api naming pattern. Currently going with rake-esque, though order is tough to get right:

  • chef:cookbook:upload
  • chef:upload:cookbook
  • chef:sync:cookbooks
  • echo
  • ... etc

CLI can fail to receive responses

Due to the Pub/Sub nature of the Server talking back to a Client/CLI, it's possible that the CLI will not receive an important piece of information and hang (ZeroMQ Pub/Sub, subscribers may not receive the first message sent on connecting). If this continues to be a problem I may change the Client->Server pipe to be Router/Router and send responses directly back to the Client in question vs Pub/Sub.

Server Client Registry

Server needs to improve it's client knowledge:

  • When a Client last checked in
  • If a Client is checking in again (currently duplicates)

Eventually, persist this info?

Also send back this extra information in ListClients response.

Default CLI Identity to ENV["USER"]

The CLI identity should be something that in most all cases won't be confused as proper client. Starting with the current user's username is a good start I believe.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.