Code Monkey home page Code Monkey logo

robo's Introduction

Robo

Build Status

Simple YAML-based task runner written in Go.

Features

  • Not super slow
  • Not super obscure
  • No dependencies
  • Variables
  • Simple

Installation

From gobinaries.com:

$ curl -sf https://gobinaries.com/tj/robo | sh

From source:

$ go get github.com/tj/robo

Usage

Command-line usage.

Listing tasks

Output tasks:

$ robo

  aws – amazon web services cli
  circle.open – open the repo in circle ci
  events – send data to the "events" topic
  push – push image from the current directory

Task help

Output task help:

$ robo help events

  Usage:

    events [project-id] [rate]

  Description:

    send data to the "events" topic

  Examples:

    Send 25 events a second to gy2d
    $ robo events gy2d 25

Running tasks

Regardless of task type (shell, exec, script) any additional arguments given will be passed.

For example suppose you have the following task:

aws:
  exec: ssh tools aws

You may then interact with the AWS cli as you would normally:

$ robo aws help
$ robo aws ec2 describe-instances

Configuration

Task configuration.

Commands

The most basic task simply runs a shell command:

hello:
  summary: some task
  command: echo world

You may also define multi-line commands with YAML's |:

hello:
  summary: some task
  command: |
    echo hello
    echo world

Commands are executed via sh -c, thus you may use shell features and positional variables, for example:

hello:
  command: echo "Hello ${1:-there}"

Yields:

$ robo hello
Hello there

$ robo hello Tobi
Hello there Tobi

Exec

The exec alternative lets you replace the robo image without the fork & shell, however note that shell features are not available (pipes, redirection, etc).

hello:
  summary: some task
  exec: echo hello

Any arguments given are simply appended.

Scripts

Shell scripts may be used instead of inline commands:

hello:
  summary: some task
  script: path/to/script.sh

If the script is executable, it is invoked directly, this allows you to use #!:

$ echo -e '#!/usr/bin/env ruby\nputs "yo"' > yo.rb
$ chmod +x yo.rb
$ cat > robo.yml
yo:
  summary: yo from rb
  script: yo.rb
^C
$ robo yo
yo

Script paths are relative to the config file, not the working directory.

Usage

Tasks may optionally specify usage parameters, which display upon help output:

events:
  summary: send data to the "events" topic
  exec: docker run -it events
  usage: "[project-id] [rate]"

Examples

Tasks may optionally specify any number of example commands, which display upon help output:

events:
  summary: send data to the "events" topic
  exec: docker run -it events
  usage: "[project-id] [rate]"
  examples:
    - description: Send 25 events a second to gy2d
      command: robo events gy2d 25

Variables

Robo supports variables via the text/template package. All you have to do is define a map of variables and use {{ }} to refer to them.

Here's an example:

stage:
  summary: Run commands against stage.
  exec: ssh {{.hosts.stage}} -t robo

prod:
  summary: Run commands against prod.
  exec: ssh {{.hosts.prod}} -t robo

variables:
  hosts:
    prod: bastion-prod
    stage: bastion-stage

The variables section does also interpolate itself with its own data via {{ .var }} and allows shell like command expressions via $(echo true) to be executed first providing the output result as a variable. Note that variables are interpolated first and then command expressions are evaluated. This will allow you to reduce repetitive variable definitions and declarations.

hash:
  summary: echos the git {{ .branch }} branch's git hash
  command: echo {{ .branch }} {{ .githash }}

variables:
  branch: master
  githash: $(git rev-parse --short {{ .branch }})

Along with your own custom variables, robo defines the following variables:

$ robo variables

    robo.file: /Users/amir/dev/src/github.com/tj/robo/robo.yml
    robo.path: /Users/amir/dev/src/github.com/tj/robo

    user.home: /Users/amir
    user.name: Amir Abushareb
    user.username: amir

Environment

Tasks may define env key with an array of environment variables, this allows you to re-use robo configuration files, for example:

// aws.yml
dev:
  summary: AWS commands in dev environment
  exec: aws
  env: ["AWS_PROFILE=eng-dev"]

stage:
  summary: AWS commands in stage environment
  exec: aws
  env: ["AWS_PROFILE=eng-stage"]

prod:
  summary: AWS commands in prod environment
  exec: aws
  env: ["AWS_PROFILE=eng-prod"]

You can also override environment variables:

$ cat > robo.yml
home:
  summary: overrides $HOME
  exec: echo $HOME
  env: ["HOME=/tmp"]
^C
$ robo home // => /tmp

Variables can also be used to set env vars.

$ cat > robo.yml
aws-stage:
  summary: AWS stage
  exec: aws
  env: ["AWS_PROFILE={{.aws.profile}}"]
variables:
  aws:
    profile: eng-stage
^C
$ robo aws-stage ...

Note that you cannot use shell featurs in the environment key.

Setup / Cleanup

Some tasks or even your entire robo configuration may require steps upfront for setup or afterwards for a cleanup. The keywords before and after can be embedded into a task or into the overall robo configuration. It has the same executable syntax as a task: script, exec and command. Defining it on a task level causes the steps to be executed before (respectively after) the task. Global before or after steps are invoked for every task in the configuration. All steps get interpolated the same way tasks and variables are interpolated.

before:
  - command: echo "global before {{ .foo }}"
after:
  - script: /global/after-script.sh

foo:
  before:
    - command: echo "local before {{ .foo }}"
    - exec: git pull -r
  after:
    - command: echo "local after"
    - exec: git reset --hard HEAD
  exec: git status

variables:
  foo: bar

Templates

Task list and help output may be re-configured, for example if you prefer to view usage information instead of the summary:

templates:
  list: |
    {{range .Tasks}}  {{cyan .Name}} – {{.Usage}}
    {{end}}

Or perhaps something more verbose:

templates:
  list: |
    {{range .Tasks}}
      name: {{cyan .Name}}
      summary: {{.Summary}}
      usage: {{.Usage}}
    {{end}}

Global tasks

By default ./robo.yml is loaded, however if you want global tasks you can simply alias to something like:

alias segment='robo --config ~/.robo.yml'

Robo chaining

You can easily use Robo to chain Robo, which is useful for multi-environment setups. For example:

prod:
  summary: production tasks
  exec: robo --config production.yml

stage:
  summary: stage tasks
  exec: robo --config stage.yml

Or on remote boxes:

prod:
  summary: production tasks
  exec: ssh prod-tools -t robo --config production.yml

stage:
  summary: stage tasks
  exec: ssh stage-tools -t robo --config stage.yml

You can also use robo's builtin variables robo.path, for example if you put all robofiles in together:

├── dev.yml
├── prod.yml
├── root.yml
└── stage.yml

And you would like to call dev, prod and stage from root:

dev:
  summary: Development commands
  exec: robo --config {{ .robo.path }}/dev.yml

stage:
  ...

Composition

You can compose multiple commands into a single command by utilizing robo's built-in robo.file variable:

one:
  summary: echo one
  command: echo one

two:
  summary: echo two
  command: echo two

all:
  summary: echo one two
  command: |
    robo -c {{ .robo.file }} one
    robo -c {{ .robo.file }} two
$ robo all
one
two

Why?

We generally use Makefiles for project specific tasks, however the discoverability of global tasks within a large team is difficult unless there's good support for self-documentation, which Make is bad at.

I'm aware of the million other solutions (Sake, Thor, etc) but I just wanted something fast without dependencies.

License

MIT

robo's People

Contributors

cgamesplay avatar jenpet avatar michaelbeil avatar stephenmathieson avatar tj avatar yields avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

robo's Issues

K-V Pair File Support

Hey guys, what you think of a possibility to pass key-value pair files (e.g. .env) to robo which will be processed before running a certain task? It would be similar to the functionality of the currently supported envs but a bit more flexible if we can just specific a env file instead of (or additionally to) a set of env vars for task.

Especially when it comes to more complex configurations where the local environment might differ quite heavily between the dev setups / machines it becomes very messy because I currently see only two options:

  • manually setting a list of individual env vars in .bashrc; wherever
  • putting them in the robo.yml file (… and eventually checking them in ☠️ )

Am I missing something regarding robo's functionality or would you consider this a feature we could think about?

Cannot expand variable in other variable on the top of line

Thank you very much for this project!
I found a bug.

This yaml file not works.

error loading configuration: yaml: line 10: did not find expected key

I got the above error.

run:
  exec: echo {{.script}}

variables:
  root: $(echo $HOME/myproject/)
  script:  {{.root}}/index.php

However this is works.

run:
  exec: echo {{.script}}

variables:
  root: $(echo $HOME/myproject/)
  script:  index.php/{{.root}}  # swap

I'm restricted where expanding variable only after string.

Better error message on YAML syntax error

I'm using robo 0.3.0 and prepared following YAML file.

robo.yml

runsql:
  summary: Run SQL
  command: |
    # ...
    if [ ! mysql --host "$db_server_ip" --user root --password --execute << _SQL_
      # SQL...
_SQL_ ]; then # This YAML syntax error causes the error
      echo "Failed to login"
    fi

I cded to the directory where above YAML file is and run robo setup, then follwing error is shown:

$ robo runsql
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x4389dc]

goroutine 1 [running]:
github.com/tj/robo/config.New(0xc2080340f2, 0x8, 0xc208032320, 0x0, 0x0)
        /Users/tj/dev/src/github.com/tj/robo/config/config.go:29 +0x13c
main.main()
        /Users/tj/dev/src/github.com/tj/robo/main.go:40 +0x24a

goroutine 2 [runnable]:
runtime.forcegchelper()
        /usr/local/Cellar/go/1.4.1/libexec/src/runtime/proc.go:90
runtime.goexit()
        /usr/local/Cellar/go/1.4.1/libexec/src/runtime/asm_amd64.s:2232 +0x1

goroutine 3 [runnable]:
runtime.bgsweep()
        /usr/local/Cellar/go/1.4.1/libexec/src/runtime/mgc0.go:82
runtime.goexit()
        /usr/local/Cellar/go/1.4.1/libexec/src/runtime/asm_amd64.s:2232 +0x1

goroutine 4 [runnable]:
runtime.runfinq()
        /usr/local/Cellar/go/1.4.1/libexec/src/runtime/malloc.go:712
runtime.goexit()
        /usr/local/Cellar/go/1.4.1/libexec/src/runtime/asm_amd64.s:2232 +0x1

This error is derived from YAML syntax error commented in above YAML file.
I would like better error message.

I'm on Kubuntu 15.04 64bit

scripts are no longer relative

~/dev/src/github.com/whatever/robofiles/development/robo.yml

foo:
  summary: say foo
  script: scripts/foo.sh

~/dev/src/github.com/whatever/robofiles/development/scripts/foo.sh

echo foo

yields:

∴ robo --config ~/dev/src/github.com/whatever/robofiles/development/robo.yml foo
sh: scripts/foo.sh: No such file or directory

  error: exit status 127

looks like 04798cb broke it

Support run multiple tasks from a command

First thing, thanks about cool task runner. I'm using robo in my projects and it works well.

I just asking do you have any plan in order to support multiple tasks in command. Such as:

robo build docker
robo asset build docker
robo build optimize deploy

(with build, docker, asset, optimize, deploy are tasks I defined in robo.yml)

I frequent use it to do some customize steps when developing. Example I just change a image and want to check it in development environment, I can quick change asset and run server again without build binary and rebuild docker.

Another solution's way to caching or watching changes of steps. Build binary step will watch the source code file changes, compose/optimize assets will watch image/css/js, docker build will watch

In case you have plan to support it but didn't have time to do that. I can help you create a pull request.

Bare task names list for completion

It would be nice to have a way to list task names without colors and formatting so it could be consumed by a bash/zsh/fish completion script for the task name. Something like robo -q ?

Adding this to robo.yml:

templates:
  list: |-
    {{range .Tasks}}{{.Name}}{{ print "\n" }}{{end}}

And sourcing this bash completion:

#/usr/bin/env bash
_robo_completion() {
  if [ "${#COMP_WORDS[@]}" != "2" ]; then
    return
  fi
  COMPREPLY=($(compgen -W "$(robo)" -- "${COMP_WORDS[1]}"))
}

complete -F _robo_completion robo

Works nicely. But it's annoying adding the list template in each robo.yml file.

Installing on Appveyor (Windows) get sudo command not found.

curl -sf https://gobinaries.com/tj/robo | sh
  ==> Downloading github.com/tj/robo@master
  ==> Resolved version master to v0.5.5
  ==> Downloading binary for windows amd64
  ==> Permissions required for installation to /usr/local/bin — alternatively specify a new directory with:
  ==>   $ curl -sf https://gobinaries.com/github.com/tj/[email protected] | PREFIX=. sh
main: line 184: sudo: command not found
Command exited with code 127

Interpolation in Variables Section

Thanks for that nifty tool! It triggered me to get rid of Makefiles in smaller projects. I saw that there is a ticket for includes which I also sort of "bumped" since I think it could be a real benefit when re-using common parts in several projects.

What do you think about the possibility to interpolate the values of the variable section? A construct like the following snippet would definitely help for more flexibility.

task:
   command: echo {{ .bar }} // Hello World e9dsasdh1

variables:
   hash: $(git rev-parse --short HEAD)
   foo: Hello
   bar: {{ .foo }} World {{ .hash }}

If I need those variables across multiple tasks it would be great if I could just reuse them from the "global scope" instead of exporting the via env vars first with a separate task. Am I missing something in the current feature set or would this be also beneficial for others?

How to nest variables

Trying to nest variable in another variable but seems not working. How should I handle such case in robo?

deploy:
  summary: deploy longhorn system
  exec: kapp deploy {{.kapp.flags}}

delete:
  summary: delete longhorn system
  exec: kapp delete {{.kapp.flags}}

port-forward:
  summary: port-forward service to localhost
  exec:	kubectl port-forward -n longhorn-system svc/longhorn-frontend 8080:80

variables:
  kapp:
    namespace: kapp
    app: longhorn-system
    flags: --namespace {{.kapp.namespace}} --app {{.kapp.app}} --diff-changes
❯ robo variables

    kapp.app: longhorn-system
    kapp.flags: --namespace {{.kapp.namespace}} --app {{.kapp.app}} --diff-changes
    kapp.namespace: kapp

pre-command / depends_on?

Like, should robo prebuild be run before robo build? Probably controversial, though.

Although, the concept like hooks might be nice.

Although this can easily be substitute by YAML templating.

includes

useful for importing sensitive/user-specific variables primarily, user templates too

RunScript should not use sh for scripts with the executable bit set

I have a script (myscript.sh) with a #!/usr/bin/env bash shebang set at the top.

RunScript runs this as sh myscript.sh, which means the shebang gets ignored, and we don't run the script using bash, which was pretty unexpected.

RunScript could work around this by calling os.Stat(filename).Mode() on the script in question, and then checking whether mode.IsDir() == false && mode & 0100 == 1, to see whether the current user can execute the script.

This would probably be a significant behavior change for current users, however.

using on windows, mac and linux

Hi,

I'm trying to find some way to use command line for some tasks that will work the same way on windows, mac and linux to help some users and even new technician to work on command line. The idea is to provide a simpler tutorial that will work on both platforms.

Example: To install a program that exists on both OSs, each of provide a different set of commands to do that. Using robo, maybe will be a good solution. Lets say you want to install atom.

  • on windows: choco install atom
  • on mac: brew install atom
  • on linux: sudo apt install atom
  • with robo: robo install atom

I would create a global robo.yml for each platform (better yet change robo code to identify the OS and allow to provide different commands for each OS)

But by new, I'm trying to use it as is and I'm facing with some issues:

  1. command: don't work well with params
  2. exec: don't work, return not support by windows

Would you (or someone) be willing to work on this changes/fixes? I would do it myself, but I'm not actually a developer and didn't know how to make this changes here. I didn't try other solutions as most of then are on python or ruby and it would need a runtime enviroument. What I liked in robo is that it is in go and it is a single binary.

Thanks and if I can help in anything to acomplish this changes, please, let me know.

******* simple ideia of a robo.yml file to allow multi-OS commands ******

#when the command is the same for all platforms
echo:
   summary: show a text
   exec: echo
   usage: "[text]" 
#when the command is different
install:
   summary: install some program
   exec-linux: sudo apt install
   exec-mac brew apt install
   exec-windows: choco install
   usage: "[program]"

show-file:
   summary: show file content
   exec: cat
   exec-windows: type
   usage: "[file]"

**** how I'm using and what don't work ****

echo:
  summary: display a text
  command: echo
  usage: "[text]"
on both it don't show anything, sounds like it doesn't receive the parameter

echo:
  summary: display a text
  exec: type 
  usage: "[text]"

on windows it returns: error: not support by windows

task specific vars

so they're well-defined and defaults are clearly visible without digging into scripts or examples.

Feature Request - Interpolate variables in examples too

It would be good to have interpolation of variables in examples too

variables:
  iac: terraform

hello:
  summary: some task
  command: |
    echo which {{ .iac }}
    which {{ .iac }}
  examples: # robo help tf-plan-mod
    - description: Find path of {{ .iac }}
      command: which {{ .iac }}

This results in this

✦ ❯ robo help hello

  Usage:

    hello 

  Description:

    some task

  Examples:
  
    Find path of {{ .iac }}
    $ which {{ .iac }}

I would like is

✦ ❯ robo help hello

  Usage:

    hello 

  Description:

    some task

  Examples:
  
    Find path of terraform
    $ which terraform

(real) Arguments?

It'd be great to be able to supply values for variables as arguments

speak:
  summary: Tell me something
  exec: echo {{word}}
$ robo speak word:hello
hello

Are you averse to this for any reason? I know you were going for simplistic, but it seems like quite a lot of power for minimal complication, especially if you somehow piggyback it off the existing variable system

task names complement

Hello.

I want task names complement like the following:

you:
  command: echo you
me:
  command: echo me
mycat:
  command: echo mycat
$ robo y
you
$ robo me
me
$ robo my
mycat

Task Alias

staging:
  summary: ...
  command: ...
  alias: ["stage"]

HTTP API?

Just a thought, but if robo had http server/api it would be super easy to do robo -> slack integration or robo -> something-else integration, removing a lot of duplicate code.

Maybe just a http: true would include the task in the HTTP API?
Let me know what you think, more than willing to implement that if you think it's a good idea.

Calling dependent tasks

Is this the idiomatic way of calling dependent tasks?

robo.yml:

one:
  command: echo one

two:
  command: echo two

all:
  command: |
    robo one
    robo two

So, calling robo again with the sub task

Nothing like make where you declare the dependency directly?

makefile:

one:
    echo one

two:
    echo two

all: one two

Just checking if I am doing it right, thanks.

Feature Request - Optionally Print the command before running a task and maybe some

I think it will be a good idea to optionally print the command before running a task to make the output a bit easier to read. I am proposing to make this optional and the command in some case could contain secrets too.

Also, it would be great if we can some line breaks or something more readable between tasks. As of now, reading logs is a bit difficult as there is no distinction in the logs.

Alternate solution:
use an echo statement in the command itself

hello:
  summary: some task
  command: |
    echo which terraform
    which terraform

Feature Request - Autocomplete

It would be good to have an autocomplete feature for task names so that one can type robo and then tab to choose the task.

robo is calling incorrect bash file when absolute paths are used

This is the robo.yml file:

bak:
  summary: backup data
  script: /mnt/backup/sh/backup_data.sh

when I run robo bak,
it prints:

error: stat /home/coder/mnt/backup/sh/backup_data.sh: no such file or directory

robo must call:
/mnt/backup/sh/backup_data.sh

and not:
/home/coder/mnt/backup/sh/backup_data.sh

robo is converting all paths to relative path and ignores absolute path

Robo abandons children

(Sorry, I couldn't resist that issue name.) Hi, I'm a big fan of robo, and I've been using it increasingly over the past year or so. I've recently noticed that robo will return control to the shell immediately when it receives a SIGINT. This is a problem, because child processes of robo may still be running cleanup behavior. The upshot of this is that any program which has cleanup behavior on exit will still be running in the background, and if it produces any output (or changes terminal modes), it will be printed in the middle of the user's shell prompt.

For a quick demo showing this behavior, see this gist. I added a Makefile to compare the behavior against make, which I would argue is the correct behavior: it catches the signal, stops running new commands, waits for all child processes to exit, and only then exits itself.

In order to bring robo inline with make, I think Robo just needs to ignore the relevant signals. The child processes are already delivered the signal as well, so the cmd.Run() call will terminate naturally. I think the proper signals to catch are SIGINT and SIGTERM. There's a stylistic question about where this behavior should live. My suggestion is cli.Run, since it's a global effect, and putting it in Task.Run feels like a violation of the Task encapsulation.

Would a PR implementing this behavior be accepted?

Exec does not escape properly

RunExec uses strings.Fields to find the binary to pass to exec.LookPath, but if I pass in a value like "/Library/Application Support/Postgres/bin/whatever" it should call LookPath with the full string, not with "Library/Application".

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.