Code Monkey home page Code Monkey logo

arbiter's Introduction

Arbiter Build Status

Arbiter Has Been Archived and is No Longer Maintained

Arbiter is a utility for generating XML Oozie workflows from a YAML specification.

Read the blog post that introduced Arbiter on Code as Craft, Etsy's engineering blog.

Mailing List

There is a mailing list for this project at https://groups.google.com/forum/#!forum/arbiter-oozie. If you have questions or suggestions for the project send them here!

Features

  1. Configuration driven: The mapping of the YAML workflow definition to the generated XML is highly configurable.
  2. Automatic dependency resolution and parallelism: Arbiter workflows are specified in terms of the dependencies between actions, rather than requiring the author to manually specify the flow. Arbiter will order the workflow actions to satisfy the dependencies as well as insert fork/join pairs to run actions in parallel when possible.

Building

Arbiter requires at least Java 7.

Arbiter is built with Maven. Run mvn clean package to build an uber-JAR suitable for use in running Arbiter.

Pre-built Arbiter JAR files are also available in Maven Central:

<dependency>
  <groupId>com.etsy</groupId>
  <artifactId>arbiter</artifactId>
  <version>0.5.0</version>
</dependency>

Usage

Before writing workflows with Arbiter, you must define at least one configuration file. See Configuration for details on writing a configuration file.

See Workflow Definition for details on writing workflows with Arbiter.

Command Line Options

Flag Meaning
-c Specifies the path to a configuration file. Required once, but can be specified multiple times to provide multiple configuration files.
-l Specifies the path to a low-priority configuration file. Low priority configurations will be overridden by standard configurations if they define overlapping settings. Optional.
-i Specifies the path to a YAML workflow definition to process. Required once, but can be specified multiple times to process multiple workflows.
-o Specifies the directory to which Arbiter will output the generated workflows. Required.
-g [] Enables generating a image of the workflow graph using Graphviz. The dot tool must be installed and on the PATH for this to work. SVG is the default format but any format supported by dot may be specified as an argument to for this flag.
-h Prints a usage message

Running Arbiter

First build an Arbiter uber-JAR as described in the Building section above. Arbiter can then be invoked like so:

java -jar arbiter.jar [OPTIONS]

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.