This sample shows how to build a ChatGPT like application in Spring and run on Azure Spring Apps. It enables ChatGPT to use your private data to answer the questions.

1. Query flow (Web API)
   1. Convert the user's query text to an embedding.
   2. Query Top-K nearest text chunks from the vector store (by cosine similarity).
   3. Populate the prompt template with the chunks.
   4. Call to OpenAI text completion API.
2. Indexing flow (CLI)
   1. Load the documents from the local disk / Azure storage.
   2. Split the text into chunks.
   3. Convert text chunks into embeddings
   4. Save the embeddings into Vector Store

You can provision Azure resources and run this application on Azure with Azure Developer CLI (AZD), which can help you build and run your application in the cloud quickly without running multiple az cli commands step by step. For more information, see Azure Developer CLI.

Let's jump in and get this up and running in Azure. When you are finished, you will have a fully functional chatgpt app deployed to the cloud. In later steps, you'll see how to setup a pipeline and run the application.

^{Screenshot of the deployed chatgpt app}

The following prerequisites are required to use this application. Please ensure that you have them all installed locally.

• Azure Developer CLI
• Java 17 or later
• Azure CLI
• An Azure subscription with access granted to Azure OpenAI (see more here)
• Powershell 7 if you use windows

To learn how to get started with any template, follow the steps in this quickstart with this template(Azure-Samples/spring-chatgpt-sample).

This quickstart will show you how to authenticate on Azure, initialize using a template, provision infrastructure and deploy code on Azure via the following commands:

# Log in to azd. Only required once per-install.
azd auth login

# Enable Azure Spring Apps feature for AZD. Only required once per-install.
azd config set alpha.springapp on

# First-time project setup. Initialize a project in the current directory, using this template. 
azd init --template Azure-Samples/spring-chatgpt-sample

# Provision and deploy to Azure
azd up

The template by default uses a pre-built documentation vector store of the public documents of the Azure Spring Apps.

If you want to load your own documents to the vector store, you can use the following command before running azd up:

# under the root of the project
mvn clean package
java -jar spring-chatgpt-sample-cli/target/spring-chatgpt-sample-cli-0.0.1-SNAPSHOT.jar --from=/<path>/<to>/<your>/<documents> --to=doc_store.json

This application utilizes the following Azure resources:

• Azure Spring Apps to host the application
• Azure OpenAI for ChatGPT
• Azure Storage as the ASA file volume for the vector store to achieve persistent storage

Here's a high level architecture diagram that illustrates these components. Notice that these are all contained within a single resource group, that will be created for you when you create the resources.

This template provisions resources to an Azure subscription that you will select upon provisioning them. Please refer to the Pricing calculator for Microsoft Azure and, if needed, update the included Azure resource definitions found in infra/main.bicep to suit your needs.

This template is structured to follow the Azure Developer CLI. You can learn more about azd architecture in the official documentation.

At this point, you have a complete application deployed on Azure. But there is much more that the Azure Developer CLI can do. These next steps will introduce you to additional commands that will make creating applications on Azure much easier. Using the Azure Developer CLI, you can delete the resources easily.

• azd down - to delete all the Azure resources created with this template

The Azure Developer CLI includes many other commands to help with your Azure development experience. You can view these commands at the terminal by running azd help. You can also view the full list of commands on our Azure Developer CLI command page.

As an alternative to AZD, you can run this sample app using the Azure CLI by following these steps.

• JDK 17
• Maven
• Azure CLI
• An Azure subscription with access granted to Azure OpenAI (see more here)

1. Use the following commands to define variables for this quickstart with the names of your resources and desired settings:

   LOCATION="eastus"
   RESOURCE_GROUP="<resource-group-name>"
   MANAGED_ENVIRONMENT="<Azure-Container-Apps-environment-name>"
   SERVICE_NAME="<Azure-Spring-Apps-instance-name>"
   APP_NAME="<Spring-app-name>"
   OPENAI_RESOURCE_NAME="<Azure-OpenAI-resource-name>"

2. Use the following command to create a resource group:

   az group create \
       --resource-group ${RESOURCE_GROUP} \
       --location ${LOCATION}

3. An Azure Container Apps environment creates a secure boundary around a group of applications. Apps deployed to the same environment are deployed in the same virtual network and write logs to the same log analytics workspace. For more information, see Log Analytics workspace overview. Use the following command to create the environment:

   az containerapp env create \
       --resource-group ${RESOURCE_GROUP} \
       --name ${MANAGED_ENVIRONMENT} \
       --location ${LOCATION} \
       --enable-workload-profiles

4. Use the following command to create a variable to store the environment resource ID:

   MANAGED_ENV_RESOURCE_ID=$(az containerapp env show \
       --resource-group ${RESOURCE_GROUP} \
       --name ${MANAGED_ENVIRONMENT} \
       --query id \
       --output tsv)

5. Use the following command to create an Azure Spring Apps service instance. An instance of the Azure Spring Apps Standard consumption and dedicated plan is built on top of the Azure Container Apps environment. Create your Azure Spring Apps instance by specifying the resource ID of the environment you created.

   az spring create \
       --resource-group ${RESOURCE_GROUP} \
       --name ${SERVICE_NAME} \
       --managed-environment ${MANAGED_ENV_RESOURCE_ID} \
       --sku standardGen2 \
       --location ${LOCATION}

1. Run the following command to create an Azure OpenAI resource in the the resource group.

   az cognitiveservices account create \
      -n ${OPENAI_RESOURCE_NAME} \
      -g ${RESOURCE_GROUP} \
      -l ${LOCATION} \
      --kind OpenAI \
      --sku s0 \
      --custom-domain ${OPENAI_RESOURCE_NAME}   

2. Create the model deployments for text-embedding-ada-002 and gpt-35-turbo in your Azure OpenAI service.

   az cognitiveservices account deployment create \
      -g ${RESOURCE_GROUP} \
      -n ${OPENAI_RESOURCE_NAME} \
      --deployment-name text-embedding-ada-002 \
      --model-name text-embedding-ada-002 \
      --model-version "2"  \
      --model-format OpenAI \
      --scale-settings-scale-type "Standard"
   
    az cognitiveservices account deployment create \
      -g ${RESOURCE_GROUP} \
      -n ${OPENAI_RESOURCE_NAME} \
      --deployment-name gpt-35-turbo \
      --model-name gpt-35-turbo \
      --model-version "0301"  \
      --model-format OpenAI \
      --scale-settings-scale-type "Standard"     

1. Run git clone https://github.com/Azure-Samples/spring-chatgpt-sample.git
2. Run cd spring-chatgpt-sample.
3. Run cp env.sh.sample env.sh and substitute the placeholders.
4. Build with mvn clean package.

Before running the web app, you need to preprocess the documents and load them into the vector store:

source env.sh
java -jar spring-chatgpt-sample-cli/target/spring-chatgpt-sample-cli-0.0.1-SNAPSHOT.jar --from=/<path>/<to>/<your>/<documents> --to=doc_store.json

Or dowload the pre-built vector store of the public documents of the Azure Spring Apps.

To run the demo in the local machine, please follow these steps:

1. Launch the web app

   source env.sh
   java -jar spring-chatgpt-sample-webapi/target/spring-chatgpt-sample-webapi-0.0.1-SNAPSHOT.jar

2. Open http://localhost:8080 in your browser.

1. Use the following command to specify the app name on Azure Spring Apps and to allocate required resources:

   az spring app create \
      --resource-group ${RESOURCE_GROUP} \
      --service ${SERVICE_NAME} \
      --name ${APP_NAME} \
      --cpu 2 \
      --memory 4Gi \
      --min-replicas 2 \
      --max-replicas 2 \
      --assign-endpoint true

2. Create a Azure storage account and a file share, then add the storage link in the Azure Container Apps environment by using the following commands. The az containerapp env storage set command creates a link between the environment and the file share.

   STORAGE_ACCOUNT_NAME="<storage-account-name>"
   FILE_SHARE_NAME="vectorstore"
   STORAGE_MOUNT_NAME="vectorstore"
   
   az storage account create \
      --resource-group ${RESOURCE_GROUP} \
      --name ${STORAGE_ACCOUNT_NAME} \
      --kind StorageV2 \
      --sku Standard_LRS \
      --enable-large-file-share \
      --output none   
   
   az storage share-rm create \
      --resource-group ${RESOURCE_GROUP} \
      --storage-account ${STORAGE_ACCOUNT_NAME} \
      --name ${FILE_SHARE_NAME} \
      --quota 1024 \
      --enabled-protocols SMB \
      --output none
   
   STORAGE_ACCOUNT_KEY=$(az storage account keys list \
      --resource-group ${RESOURCE_GROUP} \
      --account-name ${STORAGE_ACCOUNT_NAME} \
      --query "[0].value" \
      -o tsv)
   
   az containerapp env storage set \
      --resource-group ${RESOURCE_GROUP} \
      --name ${MANAGED_ENVIRONMENT} \
      --storage-name ${STORAGE_MOUNT_NAME} \
      --azure-file-account-name ${STORAGE_ACCOUNT_NAME} \
      --azure-file-account-key ${STORAGE_ACCOUNT_KEY} \
      --azure-file-share-name ${FILE_SHARE_NAME} \
      --access-mode ReadOnly

3. Add the persistent storage to the app by using the following command:

   az spring app append-persistent-storage \
      --resource-group ${RESOURCE_GROUP} \
      --service ${SERVICE_NAME} \
      --name ${APP_NAME} \
      --persistent-storage-type AzureFileVolume \
      --mount-path /opt/spring-chatgpt-sample \
      --storage-name ${STORAGE_MOUNT_NAME}

4. Upload the vector store file to the Azure storage account built in the previous step.

   az storage file upload -s ${FILE_SHARE_NAME} --source ./doc_store.json

5. Use the following command to deploy the .jar file for the app:

   az spring app deploy \
      --resource-group ${RESOURCE_GROUP} \
      --service ${SERVICE_NAME} \
      --name ${APP_NAME} \
      --artifact-path spring-chatgpt-sample-webapi/target/spring-chatgpt-sample-webapi-0.0.1-SNAPSHOT.jar \
      --env AZURE_OPENAI_ENDPOINT=<your_azure_openai_endpoint> \    
            AZURE_OPENAI_APIKEY=<your_api_key> \
            AZURE_OPENAI_CHATDEPLOYMENTID=gpt-35-turbo \
            AZURE_OPENAI_EMBEDDINGDEPLOYMENTID=text-embedding-ada-002 \
            VECTORSTORE_FILE=/opt/spring-chatgpt-sample/doc_store.json \
      --runtime-version Java_17

If you have any feature requests, issues, or areas for improvement, please file an issue. To keep up-to-date, ask questions, or share suggestions, join our GitHub Discussions. You may also contact us via [email protected].