Retrieval-augmented generation assistant demo using LangChain and Streamlit.

Setup a virtual environment in the project directory:

python3.8 -m venv .venv
source .venv/bin/activate  # Linux/MacOS
# .\.venv\Scripts\activate.bat  # Windows CMD
# .\.venv\Scripts\activate.ps1  # PowerShell
pip install -IU pip setuptools wheel

Install the RAG demo package and some extra dependencies:

# For GPU support
pip install renumics-rag[all]@git+https://github.com/Renumics/renumics-rag.git torch torchvision sentence-transformers accelerate
# For CPU support
# pip install renumics-rag[all]@git+https://github.com/Renumics/renumics-rag.git torch torchvision sentence-transformers accelerate --extra-index-url https://download.pytorch.org/whl/cpu

If you intend to edit, not simply use, this project, clone the entire repository:

git clone [email protected]:Renumics/renumics-rag.git

Then install it in editable mode.

Setup virtual environment in the project folder:

python3.8 -m venv .venv
source .venv/bin/activate  # Linux/MacOS
# .\.venv\Scripts\activate.bat  # Windows CMD
# .\.venv\Scripts\activate.ps1  # PowerShell
pip install -IU pip setuptools wheel

Install the RAG demo package and some extra dependencies:

pip install -e .[all]
# For GPU support
pip install pandas torch torchvision sentence-transformers accelerate
# For CPU support
# pip install pandas torch torchvision sentence-transformers accelerate --extra-index-url https://download.pytorch.org/whl/cpu

Install the RAG demo and some extra dependencies:

poetry install --all-extras
# Torch with GPU support
pip install pandas torch torchvision sentence-transformers accelerate
# Torch with CPU support
# pip install pandas torch torchvision sentence-transformers accelerate --extra-index-url https://download.pytorch.org/whl/cpu

Activate the environment (otherwise, prexis all subsequent commands with poetry run):

poetry shell

Note: If you have Direnv installed, you can avoid prefixing python commands with poetry run by executing direnv allow in the project directory. It will activate environment each time you enter the project directory.

If you plan to use OpenAI models, create a .env file with the following content:

OPENAI_API_KEY="Your OpenAI API key"

If you plan to use OpenAI models via Azure, create .env with the following content:

OPENAI_API_TYPE="azure"
OPENAI_API_VERSION="2023-08-01-preview"
AZURE_OPENAI_API_KEY="Your Azure OpenAI API key"
AZURE_OPENAI_ENDPOINT="Your Azure OpenAI endpoint"

If you are using Hugging Face models, a .env file is not necessary. You can configure the embeddings model, retriever and the LLM in the config file (settings.yaml). The default settings (taken from settings file are:

llm_type: 'openai'       # 'openai', 'hf' or 'azure'
llm_name: 'gpt-3.5-turbo'

relevance_score_fn: 'l2'
k: 20
search_type: 'similarity'
score_threshold: 0.5
fetch_k: 20
lambda_mult: 0.5

embeddings_model_type: 'openai'     # 'openai', 'hf' or 'azure'
embeddings_model_name: 'text-embedding-ada-002'

You can adapt it without cloning the repository by setting up an environment variable RAG_SETTINGS pointing to your local config file. You can also configure it from the GUI during the question and answering sessions. But it's important to choose the desired embeddings model because the indexing is done beforehand.

You can skip this section download the demo database with embeddings of a Formula One Dataset. This dataset is based on articles from Wikipedia and is licensed under the Creative Commons Attribution-ShareAlike License. The original articles and a list of authors can be found on the respective Wikipedia pages. To use your own data create a new data/docs directory within the project and place your documents in there (recursive directories are supported).

Note: at the moment, only HTML files can be indexed but it can be adjusted in the create-db script, this requires the ⚒️ Local Setup

Begin the process by indexing your documents. Execute the following command:

create-db

This will create a db-docs directory within the project consisting of indexed documents. To index additional documents, use the --exist-ok and --on-match flags (refer to create-db --help for more information).

Now, you can leverage the indexed documents to answer questions.

To only retrieve relevant documents via command line interface:

retrieve "Your question here"
# QUESTION: ...
# SOURCES: ...

To answer a question based on the indexed documents:

answer "Your question here"
# QUESTION: ...
# ANSWER: ...
# SOURCES: ...

To start a web application (See app --help for available application options):

app

This will open a new browser window:

You can enter your questions to get them answered by the RAG System. Each answer contains an expandable 'Sources' section that includes the text and filename for each snippet used to generate the answer.  The Settings section on the left allows you to choose different LLMs from OpenAI or Hugging Face. In the advanced settings, you can adjust the retrieval settings like the choice of relevance score function, the number of retrieved snippets, and search type. You can also change the embeddings model.  Note: Changing the embeddings model requires creating a new database with the new embeddings.

After submitting some questions, you can explore them using Renumics Spotlight by clicking the red explore button on the left:

It will open a new browser tab:

You can see a table of all questions and all document snippets on the top left side. Using the 'visible columns' button, you can control which columns are shown in the table.

On the right side is the similarity map; it shows the questions and documents based on the embeddings. You can customize it by clicking the settings button and apply coloring by 'used_by_num_question' to color the snippets by the number of questions where the retriever loaded this snippet.

You can select snippets in the similarity map or in the table to display them in the details view at the bottom. To customize the detail view, you can remove rows by clicking the 'x' and add views with the 'Add View' button. A good setup here is to display the document, used_by_num_question and type.

Exploring the exemplary F1 Dataset and over 500 questions from our RAG evaluation article can reveal interesting clusters. Document clusters of race statistics, racing techniques and strategies, and team details are frequently referenced by at least one question. Others, like Television Rights or specific race track details, are retrieved less often.

To learn more about visulization for RAG, check out our articles:

• ITNext: Visualize your RAG Data - EDA for Retrieval-Augmented Generation: visualization of single questions, answers, and related documents shows large potential for RAG. Dimensionality reduction techniques can make the embedding space accessible for users and developers

• Towards Data Science: Visualize your RAG Data - Evaluate your Retrieval-Augmented Generation System with Ragas: Utilizing UMAP-based visualization offers an interesting approach to delve deeper than just analyzing global metrics for evaluations in RAG.