Code Monkey home page Code Monkey logo

nitra's Introduction

Nitra

Nitra is a multi-process, optionally multi-server rspec and cucumber runner that uses forking to reduce memory usage and IPC to distribute builds amongst available CPUs efficiently.

Philosophy

  • Nitra attempts to do the simplest thing possible
  • Nitra (ab)uses unix primitives where possible
  • Nitra doesn't do things that unix already does better (eg. rsync)
  • IPC is accomplished via pipes and select
  • Forking is used heavily for several reasons
  • Running nitra locally should be easy
  • Running nitra on a cluster should be easy too (though verbose)
  • Config files are a nuisance and should be stuffed into rake files (deals with the verbosity)

Usage

  nitra [options] [spec_filename [...]]
      -c, --cpus NUMBER                  Specify the number of CPUs to use on the host, or if specified after a --slave, on the slave
          --cucumber [PATTERN1,PATTERN2] Full cucumber run, causes any files you list manually to be ignored.
                                         Default pattern is "features/**/*.feature".
          --cucumber-format FORMATTER    Additional cucumber format to use. Nitra will always use the "pretty" format in addition to the format defined here.
                                         Refer to the cucumber documentation for the '--format' flag for more information.
          --cucumber-out FILE            Use with --cucumber-format FORMATTER to output that formatter to a file.
          --debug                        Print debug output
      -p, --print-failures               Print failures immediately when they occur
      -q, --quiet                        Quiet; don"t display progress bar
          --rake-after-runner task:1,task:2,task:3
                                         The list of rake tasks to run, once per runner, in the runner's environment, just before the runner exits
          --rake-before-runner task:1,task:2,task:3
                                         The list of rake tasks to run, once per runner, in the runner's environment, after the runner starts
          --rake-before-worker task:1,task:2,task:3
                                         The list of rake tasks to run, once per worker, in the worker's environment, before the worker starts
          --reset                        r
                                         Reset database, equivalent to --rake-before-worker db:reset
          --slave-mode                   Run in slave mode; ignores all other command-line options
          --slave CONNECTION_COMMAND     Provide a command that executes "nitra --slave-mode" on another host
          --rspec [PATTERN1,PATTERN2]    Full rspec run, causes any files you list manually to be ignored.
                                         Default pattern is "spec/**/*_spec.rb".
          --rspec-format FORMATTER       Additional rspec formatter to use. Nitra will always use the "pretty" formatter in addition to the formatter defined here.
                                         Refer to the rspec documentation for the '--format' flag for more information.
          --rspec-out FILE               Use with --rspec-format FORMATTER to output that formatter to a file.
          --slave-mode                   Run in slave mode; ignores all other command-line options
      -e, --environment ENV              Set the RAILS_ENV to load
      -h, --help                         Show this message

Getting started

First things first add nitra to your Gemfile:

gem 'nitra'

Then run your specs locally across your cpu's cores:

bundle exec nitra --rspec

This will just run all your specs using the default number of cpu's reported by your system. Hyperthreaded intels will report a high number which might not be a good fit, you can tune this with the -c option.

Clustered commands run slightly differently. Effectively nitra will fork and exec a command that it expects will be a nitra slave, this means we can use ssh as our tunnel of choice. It looks something like this:

bundle exec nitra --rspec --slave "ssh your.server.name 'cd your/project && bundle exec nitra --slave-mode'"

When nitra --slave command it forks, execs it, and assumes it's another process that's running "nitra --slave-mode".

Running a build cluster

Nitra doesn't prescribe how you get your code onto the other machines in your cluster. For example you can run a git checkout on the boxes you want to build on if it's the fastest way for you. For our part - we've had the most success with rsync.

Our build is run via rake tasks so we end up with a bunch of generated code to rsync files back and forth - here's a basic version that might help get you up and running:

namespace :nitra do
  task :config do
      @servers = [
        {:name => "server1", :port => "66666", :c => "4"},
        {:name => "server2", :port => "99999", :c => "2"},
        {:name => "server3", :port => "77777", :c => "8"},
        {:name => "server4", :port => "88888", :c => "4"}
      ]
  end

  desc "Sync the local directory and install the gems onto the remote servers"
  task :prep => :config do
    @servers.each do |server|
      fork do
        system "ssh -p #{server[:port]} #{server[:name]} 'mkdir -p nitra/projects/your_project'"
        exec %(rsync -aze 'ssh -q -p #{server[:port]}' --exclude .git --delete ./ #{server[:name]}:nitra/projects/your_project/ && ssh -q -t -p #{server[:port]} #{server[:name]} 'cd nitra/projects/your_project && bundle install --quiet --path ~/gems')
      end
    end
    Process.waitall
  end

  task :all => :config do
    cmd = %(bundle exec nitra -p -q --rspec --cucumber --rake-before-worker db:reset)
    @servers.each do |server|
      cmd << %( --slave "ssh -p #{server[:port]} #{server[:name]} 'cd nitra/projects/your_project && bundle exec nitra --slave-mode'" -c#{server[:c]})
    end
    system cmd
  end
end

Copyright

Copyright 2012-2013 Roger Nesbitt, Powershop Limited, YouDo Limited. MIT licence.

nitra's People

Contributors

aclemons avatar anujflux avatar ben-scully avatar dylanmckay avatar jemmyw avatar krasio avatar mkrfowler avatar mogest avatar mrwinton avatar newportandy avatar proglottis avatar rhysm avatar willbryant avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.