Rubrix is a free and open-source tool for exploring and iterating on data for artificial intelligence projects.

Rubrix focuses on enabling novel, human in the loop workflows involving data scientists, subject matter experts and ML/data engineers.

With Rubrix, you can:

• Monitor the predictions of deployed models.
• Label data with a novel search-guided, iterative workflow.
• Iterate on ground-truth and predictions to debug, track and improve your data and models over time.
• Build custom dashboards on top of your model predictions and labels.

Rubrix is composed of:

• a Python library to bridge data and models, which you can install via pip.
• a web application to explore and label data, which you can launch using Docker or directly with Python.

This is an example of Rubrix's labeling mode:

And this is an example for logging model predictions from a 🤗 transformers text classification pipeline:

from datasets import load_dataset
import rubrix as rb

model = pipeline('zero-shot-classification', model="typeform/distilbert-base-uncased-mnli")

dataset = load_dataset("ag_news", split='test[0:100]')

# Our labels are: ['World', 'Sports', 'Business', 'Sci/Tech']
labels = dataset.features["label"].names

for record in dataset:
    prediction = model(record['text'], labels)

    item = rb.TextClassificationRecord(
        inputs={"text": record["text"]},
        prediction=list(zip(prediction['labels'], prediction['scores'])),
        annotation=labels[record["label"]]
    )

    rb.log(item, name="ag_news_zeroshot")

To get started you need to follow three steps:

1. Install the Python client
2. Launch the web app
3. Start logging data

You can install the Python client with pip:

pip install rubrix

There are two ways to launch the webapp:

• Using docker-compose (recommended).
• Executing the server code manually

Create a folder:

mkdir rubrix && cd rubrix

and launch the docker-contained web app with the following command:

wget -O docker-compose.yml https://git.io/rb-docker && docker-compose up

This is the recommended way because it automatically includes an Elasticsearch instance, Rubrix's main persistence layer.

When executing the server code manually you need to provide an Elasticsearch instance yourself.

1. First you need to install Elasticsearch (we recommend version 7.10) and launch an Elasticsearch instance. For MacOS and Windows there are Homebrew formulae and a msi package, respectively.
2. Install the Rubrix Python library together with its server dependencies:

pip install rubrix[server]

3. Launch a local instance of the Rubrix web app

python -m rubrix.server

By default, the Rubrix server will look for your Elasticsearch endpoint at http://localhost:9200. If you want to customize this, you can set the ELASTICSEARCH environment variable pointing to your endpoint.

The following code will log one record into the example-dataset dataset:

import rubrix as rb

rb.log(
    rb.TextClassificationRecord(inputs="my first rubrix example"),
    name='example-dataset'
)

BulkResponse(dataset='example-dataset', processed=1, failed=0)

If you go to your Rubrix app at http://localhost:6900/, you should see your first dataset.

Congratulations! You are ready to start working with Rubrix with your own data.

To better understand what's possible take a look at Rubrix's Cookbook

As a new open-source project, we are eager to hear your thoughts, fix bugs, and help you get started. Feel free to use the Discussion forum or the Issues and we'll be pleased to help out.