Code Monkey home page Code Monkey logo

do's Introduction

Do

If you and your team use Slack, and still SSH into servers to complete routine tasks, Do will make your life easier.

🚨🚨 Note! Not everything here works yet- we're working towards a 1.0 release right now, and this README describes Do as of 1.0. Check out the 1.0.0 milestone for insight on what's currently outstanding. Feel free to contribute!

Demo

Do is a very friendly bot who can run command sequences on remote systems on-demand, and drop the output in Slack when it's done.

Why Do?

  • Because sometimes, you don't need a CD system to build every time someone pushes.
  • Because sometimes, you'd rather pull out your own teeth than have to look up the syntax for that DB export again.
  • Because sometimes, you wish your project manager could self-serve staging tasks.
  • Because sometimes, "is that process still running?".
  • Because sometimes, maybe, you should back everything up before testing that thing.

Do lets you define sequences of shell commands and arbitrary functions that automate the routine and mundane, and pipes the output back into Slack- where your entire team can see the results.

Introduction

At the core of Do is the simple concept of targets and verbs.

Verbs are sequences of commands, and targets are the things that verbs are done to.

Generally, targets will be applications like webapp, blog, or db. Verbs will be things such as build, backup, start, stop, restart, or pull.

There are few limitations about what you can teach Do to perform- it's all up to you!

Just remember to keep verb and target names lowercased, and without any whitespace in them.

Getting Started

  1. Clone this repository: git clone https://github.com/wemashinc/do.git.
  2. Install Node and NPM if you don't have them already.
  3. Install dependencies: npm install.
  4. Run the setup wizard: npm run setup.
    • The wizard will guide you through setting up things on Slack's end as well.
  5. Start Do: npm start.
    • Do will spin up a PM2 process to keep itself running in the background.
  6. Invite Do to a Slack channel so you can talk to it: /invite @[your-do-username]
  7. Confirm everything's working by talking to Do in that channel: @[your-do-username] hello.
    • Do should respond immediately.

Talking to Do

Talking to Do is simple: to run verbs, you use the natural [verb] [target] pattern.

In a channel, you must prefix your commands with your Do's username to get it's attention, like this: @do test example. In a DM with Do, you'd just say test example.

For built-in commands, you simply say [command]. Like verbs, if you're in a channel you must get your Do's attention by mentioning it.

Teaching Do

πŸ’­ Not your first rodeo? Want to get right down to business? Scroll down to "Target API Reference".

So Do is running! Great. Let's teach Do about an imaginary JavaScript web application called Helloworld whose build process we want to automate.

The build process is as follows:

  • Pull latest code: git pull
  • Get dependencies: npm install and bower install

Create a new target

First, let's create a new file in the targets/ directory.

For simplicity, we'll start with the example target as a template- so we'll make a copy of example.js called helloworld.js. Do will automatically pick this file up next time it starts, so there's no need to include it anywhere.

We'll start off by changing the name property of our new target to helloworld, the description property to 'A very complex project that runs on port 3000!', and the workingDir property to ../helloworld.

Our file now looks something like this:

module.exports = {
  verbs: [
    {
      name: 'test',
      sequence: [
        'echo hello'
      ]
    }
  ],

  name: 'helloworld',
  description: 'A very complex project that runs on port 3000!',
  workingDir: '../helloworld'
};

Looks great! Let's work on our verb next.

'test' isn't what we want to do here, so let's change the name of the verb to 'build'.

We'll also add our build commands (git pull, npm install, and bower install) to the sequence array.

module.exports = {
  verbs: [
    {
      name: 'build',
      sequence: [
        'git pull',
        'npm install',
        'bower install'
      ]
    }
  ],

  name: 'helloworld',
  description: 'A very complex project that runs on port 3000!',
  workingDir: '../helloworld'
};

Bam! We have a working target. If Helloworld really existed, we could say @do build helloworld and we'd be off to the races.

Built-in Commands

  • list: lists all targets and verbs that Do knows about
  • hello: returns some information about Do.
  • help: links you to this README!

Target API Reference

documentation in progress

Contributing

contribution guidelines in progress

Known Issues

  • It's not possible to control processes through the pm2 command. This is due to how PM2 communicates with itself- it doesn't work in the context of a child process.
    • Workaround: use the PM2 Node API in a function instead of using the PM2 shell interface (see examples above).

do's People

Contributors

shockey avatar

Stargazers

 avatar

Watchers

 avatar  avatar  avatar  avatar

do's Issues

ESLint

We should adopt the WeMash ESLint presets.

Block verbs from being invoked while they're already in progress

This is a bug newly introduced by the resolution of #1.

Running child processes asynchronously allows us to kick off the same verb twice at once, which is a bad thing: being at two places in a build pipeline at once can cause some seriously subtle headaches to end users.

We should block a verb from starting if it's already running!

Do is unresponsive while a verb is being executed

Beacuse of the way the verb runner is written (that is, synchronously), Do is unable to handle any incoming requests while it's executing a verb sequence.

This may cause problems with Slack connectivity and state when running very long verb sequences.

We should refactor the verb runner to an asynchronous shell execution API, being careful to preserve the waterfall pattern of a sequence's execution.

Support older Node versions

Currently, due to ES6 presence in the codebase, Do requires node >=6.0.0.

We can transpile ES6 on the fly in older Node versions using Babel's require hook. This may have runtime performance implications, but IMO it's better to run more slowly in older Nodes than to not run at all πŸ˜ƒ

Tests

We have plenty of testable logic in the app, and an easily mockable slackbot instance.

Let's write some tests! 😈

Tightly integrate PM2

Let's start using the PM2 API to start Do in the background, and package a local PM2 with Do so it works without needing to install PM2 globally.

Clean up data avenues

Currently, the codebase is a bit murky in regards to what data is being sent where. Problems include inconsistent variable naming, and a relative lack of comments.

First-class pm2 interface

Examples

  • @do pm2 start a: starts the a pm2 process
  • @do pm2 stop b: stops the b pm2 process
  • @do pm2 restart c: restarts the c pm2 process
  • @do pm2 list: List current processes

Improve target loader error messaging

If I forget to add a required key to a target, I get this:

Oops! There was a problem parsing targets/example.js: 
'AssertionError: expected { Object (verbs, description, ...) } to have keys 'verbs', 'name', 'description', and 'workingDir''

While this is helpful, it would be nice to be more user-friendly.

README updates

  • be more focused on end user
  • add bit about inviting bot to a room
  • need more clarity on the target api, sequences...
  • screenshots would be nice
  • developer section: overview of data flow, what reused variable names conceptually represent, etc

User trust system

Currently, anyone on a Slack team can invoke any command. This is suboptimal from a security perspective: consider a use case where Do is running on a production system.

Possible solution: a user is defined as the superuser (by handle or some other unique Slack identifier) in Do config, and commands are added to instruct Do to start and stop trusting additional users. Those users, in turn, can instruct Do to start or stop trusting any other users (except the user defined as superuser in the config).

Config wizard

We have an empty wizard.js file in the repository.

There should be a config wizard there πŸ˜„

  • Adopt sample PM2 file
  • walk user through obtaining a Slack token
  • set output channel and emoji

Create module system for handlers

Currently, handlers for help, hello, and list are hardcoded into index.js.

We should break those out into a module system similar to targets, so they're easier to reason about and easier to create and customize.

Add self-updating mechanism

It'd be awesome to be able to update do via Slack.

Do could, itself, be a target, with an update verb that runs git pull and then exits, allowing the process manager running Do to restart it.

Support command arguments

Imagine we wanted a log verb that outputs the last n lines from a process manager.

Possible formats, where n = 100:

  • @do log example 100
  • @do log example n:100

Accepting arguments may require us to augment the sequence API (which is going on in #3 for 1.0 as well).

We'll also want to sanitize the argument inputs- though we can generally trust users, we can't Trust Usersβ„’, so let's stay on the safe side.

Extend sequence API

The sequence API is kinda weird, in that you usually end up passing an array of words broken by spaces, for example ['npm', 'install'].

I've yet to see a use case that breaks that pattern, actually.

We could save users some keystrokes (and headaches) by allowing them to pass in a string (for example, npm install, which will be broken down into the command and array of arguments to be passed into exec behind the scenes.

pm2 start fails

>>> pm2 restart -s wmsv-wmffmpeg 
/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon/node_modules/amp-message/index.js:126
  return new Buffer('j:' + JSON.stringify(arg));
                                ^
TypeError: Converting circular structure to JSON
    at Object.stringify (native)
    at pack (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon/node_modules/amp-message/index.js:126:33)
    at encode (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon/node_modules/amp-message/index.js:101:14)
    at Message.toBuffer (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon/node_modules/amp-message/index.js:68:10)
    at ReqSocket.Socket.pack (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon/lib/sockets/sock.js:92:14)
    at ReqSocket.send (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon/lib/sockets/req.js:96:21)
    at Client.call (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/pm2-axon-rpc/lib/client.js:34:13)
    at Object.executeRemote (/home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/lib/Satan.js:563:23)
    at /home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/lib/CLI.js:1047:13
    at /home/ubuntu/.nvm/versions/node/v4.2.6/lib/node_modules/pm2/node_modules/async/lib/async.js:181:20

I think there may be some isolation regarding our runner process system that’s keeping the pm2 binary from communicating with its god process.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.