End-to-end open-source voice agents platform: Quickly build LLM based voice driven conversational applications

Bolna is the end-to-end open source production ready framework for quickly building LLM based voice driven conversational applications.

Bolna helps you create AI Voice Agents which can be instructed to do tasks beginning with:

1. Initiating a phone call using telephony providers like Twilio, Exotel, etc.
2. Transcribing the conversations using Deepgram, etc.
3. Using LLMs like OpenAI, Llama, Cohere, Mistral, etc to handle conversations
4. Synthesizing LLM responses back to telephony using AWS Polly, XTTS, ElevenLabs, Deepgram etc.
5. Instructing the Agent to perform tasks like sending emails, text messages, booking calendar after the conversation has ended

Refer to the docs for a deepdive into all supported providers.

A basic local setup uses Twilio for telephony. We have dockerized the setup in local_setup/. One will need to populate an environment .env file from .env.sample.

The setup consists of four containers:

1. Twilio web server: for initiating the calls one will need to set up a Twilio account
2. Bolna server: for creating and handling agents
3. ngrok: for tunneling. One will need to add the authtoken to ngrok-config.yml
4. redis: for persisting agents & prompt data

Running docker-compose up --build will use the .env as the environment file.

Once the docker containers are up, you can now start to create your agents and instruct them to initiate calls.

Once you have the above docker setup and running, you can create agents and initiate calls.

1. Refer to the official Agent API to create an agent
2. Initiate a call via API similar to Call API to receive a call

You can populate the .env file to use your own keys for providers.

In case you wish to extend and add some other Telephony like Vonage, Telnyx, etc. following the guidelines below:

1. Make sure bi-directional streaming is supported by the Telephony provider
2. Add the telephony-specific input handler file in input_handlers/telephony_providers writing custom functions extending from the telephony.py class
   1. This file will mainly contain how different types of event packets are being ingested from the telephony provider
3. Add telephony-specific output handler file in output_handlers/telephony_providers writing custom functions extending from the telephony.py class
   1. This mainly concerns converting audio from the synthesizer class to a supported audio format and streaming it over the websocket provided by the telephony provider
4. Lastly, you'll have to write a dedicated server like the example twilio_api_server.py provided in local_setup to initiate calls over websockets.

Though the repository is completely open source, you can connect with us if interested in managed hosted offerings or more customized solutions.

We love all types of contributions: whether big or small helping in improving this community resource.

1. There are a number of open issues present which can be good ones to start with
2. If you have suggestions for enhancements, wish to contribute a simple fix such as correcting a typo, or want to address an apparent bug, please feel free to initiate a new issue or submit a pull request
3. If you're contemplating a larger change or addition to this repository, be it in terms of its structure or the features, kindly begin by creating a new issue open a new issue and outline your proposed changes. This will allow us to engage in a discussion before you dedicate a significant amount of time or effort. Your cooperation and understanding are appreciated