Youtube Demo Here! ⇩

Japanese Demo Her! ⇩

This is a sample project to demonstrate how to use local LLM and TTS in Voice Agent. Using rinna/japanese-gpt-neox-3.6b-instruction-sft as LLM and Style-Bert-Vits-2 as TTS.

git clone https://github.com/teftef6220/Local_LLM_bot.git
cd Local_LLM_bot

python -m venv venv
source venv/bin/activate

Install pytorch and torchaudio and torchvision from official website

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

Install cudnn from official website

Install another requirements

pip install -r requirements.txt

Fine-tune LLM with your data.

Use Colab notebook to fine-tune LLM. You can use this notebook.

You can use

• rinna/japanese-gpt-neox-3.6b-instruction-sft
• rinna/japanese-gpt-neox-3.6b-instruction-sft-v2
• google/gemma-2b-it
• google/gemma-7b-it
• tokyotech-llm/Swallow-7b-instruct-hf
• tokyotech-llm/Swallow-13b-instruct-hf

as a pretrained model.

Fine-tune TTS with your data. You can use Style-Bert-Vits-2 as a pretrained model.

Put your fine-tuned LLM dir in llm_models directory

llm_models
    |
    |---model_instance_dir
        |
        |---adapter_model.bin
        |---adapter_model.json

and put your fine-tuned TTS dir in Voice_models directory.

Voice_models
    |
    |---model_name
        |
        |---model_name_e100_s2000.safetensors
        |---config.json
        |---style_vectors.npy

Set config in all_config.py

python llm_agent.py

Also provided a simple Bluesky bot that uses the Local LLM. You can run it with the following command.

you can mention the bot and the bot will reply to your mention.like

@latextex.bsky.social ねえ、名前教えてよ

set .env file as below

BS_USER_NAME = "your email address"
BS_PASSWORD = "your password"

and run the bot with the following command.

python blue_sky_bot.py

this bot can detect mentions and reply to your mentions use LLM.

• litagin02/Style-Bert-VITS2

• fishaudio/Bert-VITS2

This project is licensed under the Affero General Public License v3.0 - see the LICENSE file for details.