Note: This project is experimental and is not an officially supported Google product.

This project is intended to be a working demo showcasing how to extend an LLM-based application using Cloud Databases for Retrieval Augmented Generation (RAG). It's intended to both demonstrate how to best use advanced RAG patterns with Databases, as well as provide a basic outline for extending your own LLMs.

This demo showcases an "SFO Airport Assistant": a San Francisco Airport-based AI assistant that has access to information about airports, flights, and amenities. It can help answer users questions like:

• Are there any luxury shops?
• Where can I get coffee near gate A6?
• Where can I find a gift?
• What flights are headed to NYC tomorrow?

• Introduction
• Table of Contents
• Architecture
  • Overview
  • Extension as a service
• Deploying
  • Setting up your Database
  • Deploying the Extension Service
  • Running the LLM-based Application
• Writing your own extension

This demo contains 3 key parts:

1. Application -- The LLM-based app that acts as orchestrates layer for the interaction with the LLM.
2. Extension -- The extension service provides the application concrete, discrete actions that allow the LLM to interact with the Database.
3. Database -- The database containing the data the LLM can use to answer questions. For this application, the database used was intentionally designed to be interchangeable in order to make it easier to run this on your preferred database.

While it's often possible to expose similar functionality directly into your application, running your extension service has several benefits:

1. Better recall - LLMs perform better when given smaller, discrete tasks they can use to accomplish larger goals. By mapping a specific action to a specify, pre-determined query it significantly improves the LLMs ability to leverage it successfully.
2. Better scalability - Running the extension as a separate service both allows multiple different LLMs to leverage it, as well as allowing it to scale independently. It allows allows production best practices such as connection pooling or caching.
3. Better security - LLMs are susceptible to attacks such as "jailbreaking" to circumvent safety measures that are put in place. Using an intermediary service allows the application to handle authentication and authorization through more standard and secure channels.

Deploying this demo consists of 3 steps:

1. Setting up your Database -- creating your database and initializing it with data
2. Deploying your Extension -- deploying your extension service and connecting it to your database
3. Running the LLM-based application -- running your application locally

The extension service uses an interchangeable 'datastore' interface. Choose one of any of the database's listed below to set up and initialize your database:

• Set up and configure AlloyDB

Instructions for deploying the extension service

Instructions for running app locally

This demo can also serve as a starting point for writing your own extension. The directory is organized into the following folders:

You can copy or fork the extension-service folder to customize it to your needs. There are two main places you want to start:

• extension-service/app/routes.py - contains the API endpoints that the LLM will call
• extension-service/datastore/datastore.py - contains the interface used to abstract the database. There are specific implementations of this in the providers folder that can be customized with logic for your specific schema.