name | description | languages | products | page_type | urlFragment | ||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Creative Writing Assistant |
Working with Agents using Promptflow (Python Implementation) |
|
|
sample |
agent-openai-python-prompty |
This sample demonstrates how to create and work with AI agents. The app takes a topic and instruction input and then calls a research agent, writer agent, and editor agent.
In this sample we will be using the creative writting assistant to find the latest camping trends and activities in winter. The research agent
will recieve some context we provide and an instruction to find information on what we are looking for. It will use this information to create queries, which it will pass to the Bing Search API to search the web for relevant information to return. The product agent
will also recieve the context we provided and will use Azure AI Search to search through camping product information in a vector store and return the products that are semantically similar to the context.
The research and products returned will be sent to the writing agent
, along with the context and instructions we provided. The writer then uses all of this information to create an article. This article is passed to an editor agent
that analyzes the article, provides feedback for writer and decides whether to accept or reject the article. If the article is rejected the feedback is sent to the researcher and writer agents and a new article is created that incoperates the feedback. In this sample the editor can only reject the article twice. The edited article is then returned to the user.
This sample uses the Azure OpenAI to access the LLM that will drive the agents. For this sample we recommend using either GPT-35-turbo or GPT-4 with versions 1106 or 0125. You can find the regions where these versions are available here. It also leverages Prompty and Prompt Flow to create, manage and evaluate the prompts into the code. Prompty is a 'markdown-like' file with a .prompty
extension for developing prompt templates.
By the end of deploying this template you should be able to:
- Describe what Prompty and Prompt Flow provide
- Understand Agentic workflows for building LLM Apps
- Build, run, evaluate, and deploy, an AI Agent App to Azure.
This project template provides the following features:
- An
Agents
folder with all the agents mentioned in the project description. Each agent subfolder is made up of a.prompty
and.py
file. - An
orchestrator.py
file where the agent workflow is defined. requirements.txt
file with all the python packages needed to run this example.- An
app.py
file that enables you to run this application as a Flask app. - A
.env.sample
file to let you know which provisioned resources you will need to run this app.
Each template has either Managed Identity or Key Vault built in to eliminate the need for developers to manage these credentials. Applications can use managed identities to obtain Microsoft Entra tokens without having to manage any credentials.
Additionally, we have added a GitHub Action tool that scans the infrastructure-as-code files and generates a report containing any detected issues.
To ensure best practices in your repo we recommend anyone creating solutions based on our templates ensure that the Github secret scanning setting is enabled in your repos.
(Embed demo video here)
- Azure Subscription - Signup for a free account.
- Visual Studio Code - Download it for free.
- GitHub Account - Signup for a free account.
- Access to Azure Open AI Services - Learn about getting access.
- Ability to provision Azure AI Search (Paid) - Required for Semantic Ranker
- Terrafrom - Install terraform to run deployments
- Docker Desktop - Install Docker Desktop to run deployments
- Recommended Deployment Region - East US 2 is the recommened region for this deployment. Not all models and services are available for each region. Learn more here.
The repository is instrumented with a devcontainer.json
configuration that can provide you with a pre-built environment that can be launched locally, or in the cloud. You can also elect to do a manual environment setup locally, if desired. Here are the three options in increasing order of complexity and effort on your part. Pick one!
- Pre-built environment, in cloud with GitHub Codespaces
- Pre-built environment, on device with Docker Desktop
- Manual setup environment, on device with Anaconda or venv
The first approach is recommended for minimal user effort in startup and maintenance. The third approach will require you to manually update or maintain your local environment, to reflect any future updates to the repo.
To setup the development environment you can leverage either GitHub Codespaces, a local Python environment (using Anaconda or venv), or a VS Code Dev Container environment (using Docker).
This is the recommended option.
- Fork the repo into your personal profile.
- In your fork, click the green
Code
button on the repository - Select the
Codespaces
tab and clickCreate codespace...
You can also click this button:
This should open a new browser tab with a Codespaces container setup process running. On completion, this will launch a Visual Studio Code editor in the browser, with all relevant dependencies already installed in the running development container beneath. Congratulations! Your cloud dev environment is ready!
- Once you've launched Codespaces you can proceed to step 2.
This option uses the same devcontainer.json
configuration, but launches the development container in your local device using Docker Desktop. To use this approach, you need to have the following tools pre-installed in your local device:
- Visual Studio Code (with Dev Containers Extension)
- Docker Desktop (community or free version is fine)
Make sure your Docker Desktop daemon is running on your local device. Then,
- Fork this repo to your personal profile
- Clone that fork to your local device
- Open the cloned repo using Visual Studio Code
If your Dev Containers extension is installed correctly, you will be prompted to "re-open the project in a container" - just confirm to launch the container locally. Alternatively, you may need to trigger this step manually. See the Dev Containers Extension for more information.
Once your project launches in the local Docker desktop container, you should see the Visual Studio Code editor reflect that connection in the status bar (blue icon, bottom left). Congratulations! Your local dev environment is ready!
- Once you've launched your docker container environment you can proceed to step 2.
In order to run this sample locally you will need to:
If all of the above are correctly installed you can set up your local developer environment as follows.
-
First, fork the repo, and then clone the code sample locally:
git clone https://github.com/Azure-Samples/agent-openai-python-prompty.git
-
Open the repo in VS Code and navgate to the src directory
cd code . cd src
-
Install the Prompt Flow Extension in VS Code
- Open the VS Code Extensions tab
- Search for "Prompt Flow"
- Install the extension
-
Install the Azure CLI for your device OS
-
Cd into the src/api folder
cd src/api
-
Create a new local Python environment using either anaconda or venv for a managed environment.
a. Option 1: Using anaconda
```bash conda create -n agent-openai-python-prompty python=3.11 conda activate agent-openai-python-prompty pip install -r requirements.txt ```
b. Option 2: Using venv
```bash python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt ```
We setup our development ennvironment in the previous step. In this step, we'll provision Azure resources for our project, ready to use for developing our LLM Application.
Start by connecting your Visual Studio Code environment to your Azure account:
- Open the terminal in VS Code and use command
az login
. - Complete the authentication flow.
If you are running within a dev container, use these instructions to login instead:
- Open the terminal in VS Code and use command
az login --use-device-code
- The console message will give you an alphanumeric code
- Navigate to https://microsoft.com/devicelogin in a new tab
- Enter the code from step 2 and complete the flow.
In either case, verify that the console shows a message indicating a successful authentication. Congratulations! Your VS Code session is now connected to your Azure subscription!
For this project, we need to provision multiple Azure resources in a specific order. Before, we achieved this by running the provision.sh
script. Now, we'll use the Azure Developer CLI (or azd
) instead, and follow the steps below.
Visit the azd reference for more details on tool syntax, commands and options.
- If you setup your development environment manually, follow these instructions to install
azd
for your local device OS. - If you used a pre-built dev container environment (e.g., GitHub Codespaces or Docker Desktop) the tool is pre-installed for you.
- Verify that the tool is installed by typing
azd version
in a terminal.
- Start the authentication flow from a terminal:
azd auth login
- This should activate a Device Code authentication flow as shown below. Just follow the instructions and complete the auth flow till you get the
Logged in on Azure
message indicating success.Start by copying the next code: <code-here> Then press enter and continue to log in from your browser...
- Run this unified command to provision all resources. This will take a non-trivial amount of time to complete.
azd up
- On completion, it automatically invokes a
postprovision.sh
script that will attempt to log you into Azure. You may see something like this. Just follow the provided instructions to complete the authentication flow.No Azure user signed in. Please login.
- Once logged in, the script will do the following for you:
- Download
config.json
to the local device - Populate
.env
with required environment variables - Populate your data (in Azure AI Search, Azure CosmosDB)
- Create relevant Connections (for prompt flow)
- Upload your prompt flow to Azure (for deployment)
- Download
That's it! You should now be ready to continue the process as before. Note that this is a new process so there may be some issues to iron out. Start by completing the verification steps below and taking any troubleshooting actions identified.
The script should set up a dedicated resource group with the following resources:
- Azure AI services resource
- Azure Machine Learning workspace (Azure AI Project) resource
- Search service (Azure AI Search) resource
- Bing Search (Bing Search) resource
The script will set up an Azure AI Studio project with the following model deployments created by default, in a relevant region that supports them. Your Azure subscription must be enabled for Azure OpenAI access.
- gpt-3.5-turbo
- text-embeddings-ada-002
- gpt-4
The Azure AI Search resource will have Semantic Ranker enabled for this project, which requires the use of a paid tier of that service. It may also be created in a different region, based on availability of that feature.
The script should automatically create a config.json
in your root directory, with the relevant Azure subscription, resource group, and AI workspace properties defined. These will be made use of by the Azure AI SDK for relevant API interactions with the Azure AI platform later.
If the config.json file is not created, simply download it from your Azure portal by visiting the Azure AI project resource created, and looking at its Overview page.
The default sample has an .env.sample
file that shows the relevant environment variables that need to be configured in this project. The script should create a .env
file that has these same variables but populated with the right values for your Azure resources.
If the file is not created, simply copy over .env.sample
to .env
- then populate those values manually from the respective Azure resource pages using the Azure Portal (for Azure CosmosDB and Azure AI Search) and the Azure AI Studio (for the Azure OpenAI values)
Change to api/agents folder:
cd src/api
To run just the orchestrator logic:
python -m api.agents.orchestrator
To run the flask webserver:
flask --debug --app api.app:app run --port 5000
http://127.0.0.1:5000/get_article?context=Write an article about camping in alaska&instruction=find specifics about what type of gear they would need and explain in detail
In a new terminal
cd src/web
First install node packages:
npm install
Then run the web app with a local dev web server:
npm run dev
Then run evaluation
cd evaluate
python evaluate.py
Now, we need to understand how well our prompt flow performs using defined metrics like groundedness, coherence etc. To evaluate the prompt flow, we need to be able to compare it to what we see as "good results" in order to understand how well it aligns with our expectations.
We may be able to evaluate the flow manually (e.g., using Azure AI Studio) but for now, we'll evaluate this by running the prompt flow using gpt-4 and comparing our performance to the results obtained there. To do this, follow the instructions and steps in the notebook evaluate-chat-prompt-flow.ipynb
under the eval
folder.
At this point, we've built, run, and evaluated, the prompt flow locally in our Visual Studio Code environment. We are now ready to deploy the prompt flow to a hosted endpoint on Azure, allowing others to use that endpoint to send user questions and receive relevant responses.
This process consists of the following steps:
- We push the prompt flow to Azure (effectively uploading flow assets to Azure AI Studio)
- We activate an automatic runtime and run the uploaded flow once, to verify it works.
- We deploy the flow, triggering a series of actions that results in a hosted endpoint.
- We can now use built-in tests on Azure AI Studio to validate the endpoint works as desired.
Just follow the instructions and steps in the notebook push_and_deploy_pf.ipynb
under the deployment
folder. Once this is done, the deployment endpoint and key can be used in any third-party application to integrate with the deployed flow for real user experiences.
-
Login to Azure Shell
-
Follow the instructions to create a service principal here
-
Follow the instructions in steps 1 - 8 here to add create and add the user-assigned managed identity to the subscription and workspace.
-
Assign
Data Science Role
and theAzure Machine Learning Workspace Connection Secrets Reader
to the service principal. Complete this step in the portal under the IAM. -
Setup authentication with Github here
{
"clientId": <GUID>,
"clientSecret": <GUID>,
"subscriptionId": <GUID>,
"tenantId": <GUID>
}
- Add
SUBSCRIPTION
(this is the subscription) ,GROUP
(this is the resource group name),WORKSPACE
(this is the project name), andKEY_VAULT_NAME
to GitHub.
-
Follow the instructions to create a custom env with the packages needed here
- Select the
upload existing docker
option - Upload from the folder
runtime\docker
- Select the
-
Update the deployment.yml image to the newly created environemnt. You can find the name under
Azure container registry
in the environment details page.
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.