Interactively explore unstructured datasets from your dataframe.

Spotlight helps you to understand unstructured datasets fast. You can quickly create interactive visualizations and leverage data enrichments (e.g. embeddings, prediction, uncertainties) to identify critical clusters in your data.

Spotlight supports most unstructured data types including images, audio, text, videos, time-series and geometric data. You can start from your existing dataframe:

And start Spotlight with just a few lines of code:

from renumics import spotlight

spotlight.show(df, dtype={"image": spotlight.Image, "embedding": spotlight.Embedding})

Machine learning and engineering teams use Spotlight to understand and communicate on complex unstructured data problems. Here are some examples on publicly available datasets along with code snippets (👨‍💻), interactive demos (🕹️) and blog articles (📝):

Get started by installing Spotlight and loading your first dataset.

• Python version 3.8-3.11

pip install renumics-spotlight

We recommend installing Spotlight and everything you need to work on your data in a separate virtual environment.

import pandas as pd
from renumics import spotlight

df = pd.read_csv("https://renumics.com/data/mnist/mnist-tiny.csv")
spotlight.show(df, dtype={"image": spotlight.Image})

pd.read_csv loads a sample csv file as a pandas DataFrame.

spotlight.show opens up spotlight in the browser with the pandas dataframe ready for you to explore. The dtype argument specifies custom column types for the browser viewer.

import datasets
from renumics import spotlight

ds = datasets.load_dataset('renumics/emodb-enriched', split='all')
layout= spotlight.layouts.debug_classification(label='gender', prediction='m1_gender_prediction', embedding='m1_embedding', features=['age', 'emotion'])
spotlight.show(ds, layout=layout)

Here, the data types are discovered automatically from the dataset and we use a pre-defined layout for model debugging. Custom layouts can be built programmatically or via the UI.

The datasets[audio] package can be installed via pip.

We have added crash report and performance collection. We do NOT collect user data other than an anonymized Machine Id obtained by py-machineid, and only log our own actions. We do NOT collect folder names, dataset names, or row data of any kind only aggregate performance statistics like total time of a table_load, crash data, etc. Collecting Spotlight crashes will help us improve stability. To opt out of the crash report collection define an environment variable called SPOTLIGHT_OPT_OUT and set it to true. e.G.export SPOTLIGHT_OPT_OUT=true

• Open an issue on Github
• Have a coffee talk with us
• Join our channel on Discord

• 🤗 Huggingface example spaces and datasets
• 🏀 Playbook for data-centric AI workflows
• 🍰 Sliceguard library for automatic slice detection

We are currently participating in the running Hacktoberfest 2023.

If you would like to contribute to Spotlight, the easiest way is to have a look at our Contribution Docs and the CONTRIBUTING.md.

We are also equally happy about non-code contributions -- whether it's reporting bugs, suggesting features, contributing design ideas, or offering feedback, every non-code contribution is highly valued and helps make our project better for everyone.