Ordasity inspired distributed task runner.

Metafora is a Go library designed to run long-running (minutes to permanent) tasks in a cluster.

IRC: #metafora on Freenode

• Distributed - horizontally scalable, elastic
• Masterless - work stealing, not assigning, automatic rebalancing
• Fault tolerant - tasks are reassigned if nodes disappear
• Simple - few states, no checkpointing, no configuration management
• Extensible - well defined interfaces for implementing balancing and coordinating
• Exactly-once - attempts to ensure one-and-only-one instance of each submitted task is running

Many aspects of task running are left up to the Handler implementation such as checkpointing work progress, configuration management, and more complex state transitions than Metafora provides (such as Paused, Sleep, etc.).

koalemosd is a sample consumer implementation that can be run as a daemon (it requires etcd). koalemosctl is a sample command line client for submitting tasks to koalemosd.

# Install etcd as per https://github.com/coreos/etcd#getting-etcd
# Run the following in one terminal:
go get -v -u github.com/lytics/metafora/examples/koalemosd
koalemosd

# Run the client in another
go get -v -u github.com/lytics/metafora/examples/koalemosctl
koalemosctl sleep 3 # where "sleep 3" is any command on your $PATH

Q. Is it ready for production use?

Yes. Metafora with the etcd coordinator has been the production work system at Lytics since January 2014.

We're in the process of migrating more of our internal work system into Metafora.

Since Metafora is still under heavy development, you probably want to pin the dependencies to a commit hash or tag to keep the API stable. The master branch is automatically tested and is safe for use if you can tolerate API changes.

Q. Where is the metaforad daemon?

It doesn't exist. Metafora is library for you to import and use in a service you write. Metafora handles task management but leaves implementation details such as task implementation and daemonization up to the user.

Q. Why not use Ordasity?

We have an existing work running system written in Go and needed a new distribution library for it. There's over 25k lines of Go we'd like to reuse and couldn't with Ordasity as it runs on the JVM.

Q. Why not use donut?

We evaluated donut and found it far from production use. While we've been inspired by many of its basic interfaces there really wasn't much code we were interested in reusing. At ~600 lines of code in donut, starting from scratch didn't seem like it would lose us much.

That being said we're very appreciative of donut! It heavily influenced our design.

Q. Why not use goworker?

goworker does not support rebalancing and appears to be more focused on a high rate (>1/s) of short lived work items. Metafora is designed for a low rate (<1/s) of long lived work items. This means rebalancing running work is critical.

Q. Why not use a cluster management framework like Mesos or Kubernetes?

You can use a cluster management framework to run Metafora, but you shouldn't use Metafora as a cluster management framework.

While Metafora tasks are long lived, they're often not individually large or resource intensive. Cluster management frameworks' smallest unit of work tends to be an operating system process. We wanted to run many tasks per process.

Cluster management frameworks are quite large in terms of code and operational complexity -- for good reason! They're a much more powerful and general purpose tool than Metafora. Metafora is being written, deployed, and maintained by a very small team, so minimizing operational complexity and overhead is a key feature.

Q. What does metafora mean?

It's Greek for "transfer" and also refers to a winch on boats. We borrowed the Greek naval naming theme from Kubernetes.