Code Monkey home page Code Monkey logo

voiceai-asuna's Introduction

PROJECT Asuna (NOT AI/ChatGPT yet)

ASUNA SAO

Welcome to project Asuna.

(voice available in Minimize mode in chat)

Feel Free to Support Me:

Buy Me A Coffee

Description:

  • This is not a ChatGPT or full blown all knowing AI

  • This is an English only, pattern based chat bot (for now)

  • Currently using regular expressions to catch and reply specific patterns of messaged and collecting inputs and unknown inputs to train in future

  • Once, With sufficient data and resources, we will perform AI training

  • If you have any idea or want to provide IO data, please file an Issue or Pull Request

  • Documentation will be created soon for ease of group development.

  • Currently this is an one man project and my 1st language is not English, Using 1000 years of knowledge from animes and movies to enrich the chat input response.

  • So please don't expect much but I hope I'll be able to provide great performance with it.

  • Please have patience Maintaining completely new (self made) server, UI and back end is not easy task (for me). Then again adding IO based patterns, sequencing them takes time.

  • My/Our main goal is to create a Multi platform accessible voice assistant.

  • our additional plan is to create their animated avatar and make it available online (via Browsers)

Current status:

  • Create server and web UI
  • Add anime live2D
  • Connect chat
  • Server and UI improvements
  • Add intent and context
  • Understand and reply according to context
  • Add more Commands
  • Add voice recognition
  • Add voice output
  • Add anime live2D animations
  • Improve sidebars
  • Allow bot send pictures
  • Allow bot send audio
  • Allow bot send video
  • FINALLY TRAIN IT WITH GPT-2

How To Run:

  • First install the REQUIREMENTS, click it to see details
  • To Launch the server run the RUN_ME.py file
    python RUN_ME.py
    
  • Demo video coming soon...

Requirement:

  • Python 3.7 or higher
  • Works on Android Pydroid 3 ๐Ÿ˜„ too (most development is done using this)

Common IO: (similar inputs may/will work too)

  • Basic hiiii, hello
  • What's your/my name / how're u
  • Whats the time / tell time
  • Static Q/A, like whats newtons 3rd law / whos the president of canada / whats root(69+420)
  • Whats the latest news / news highlights
  • Tell me about yourself / ... your hobby/favorite game/anime
  • Love ya
  • Repeat after me -> will reply whatever you say next. Say stop/stop repeating to stop
  • change dress to change costumes and change room to switch background
  • Many more (forgot mostly) and many more coming soon

CONTRIBUTION GUIDELINE:

  • Make changes on whatever you feel like
  • Place some good comments (so that an intermediate python programmer can understand)
  • Make a PR and try to explain what you have changed and if theres any issue.
  • Keep in mind if you are interested :
    • pyrobox.py is the server main file (like django or flask)
    • App_server.py is the file that handles client-host request responses.
    • Chat_raw2.py is the tool that actually handles what msg will do what and reply what (can be used as standalone in CLI mode for development mode, will use test account)
  • Don't worry about marge-issue, I'll update the code
  • Most importantly AS A MOBILE CODER, I USUALLY DON'T FOLLOW ANY CODE STYLE GUIDELINE (or pep8), SO PLEASE DON'T WORRY ABOUT THAT TOO MUCH. (I'll try to follow it in future)

Thanks to:

  1. Reki Kawahara and abec (for creating Asuna)
  2. Sony group (for Wake me up Asuna App idea and illustrations)
  3. Pixi.js and live2D for character animation
  4. Replit for continuously hosting Demo link for free (not anymore)
  5. Render.com for hosting Demo link for free
  6. (Coming soon) Anyone who's willing to share chat data and ideas

voiceai-asuna's People

Contributors

rasan147 avatar s-b-repo avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

voiceai-asuna's Issues

Multiline msg is crashing

Screenshot_2023-02-03-17-28-37-23_c759c44d10a956b96f85cc66750ff86e

Multi line input is not yet supported but server and chat handler, but can be sent by web ui. Either disable or fix multipart form data issue

tts for the chat bot

you'll need to create an endpoint in your website's backend that takes in text input and outputs the synthesized audio. You can use the Tortoise-TTS library in your backend code to generate the audio. Here's a sample code snippet in Python:

python

from tortoiseTTS import Tortoise

def generate_audio(text):
tortoise = Tortoise()
audio = tortoise.say(text)
return audio

You can expose this function as an API endpoint using a Python web framework like Flask or Django. For example, with Flask:

python

from flask import Flask, request

app = Flask(name)

@app.route('/api/synthesize', methods=['POST'])
def synthesize():
text = request.form.get('text')
audio = generate_audio(text)
return audio

if name == 'main':
app.run()

Note that this is just a basic example, and you may need to modify it depending on your specific use case. Also, be aware that generating audio can be a computationally intensive task, so you may want to optimize the code for performance if you expect a high volume of requests.

[FEAT] add device control

Add "allow-device-control" in user.json (False)
Toggle switch in webUI
On 1st toggle user needs to agree from commandline running terminal window
Then will verify,
Verify process:

  1. Browser request for token
  2. Server generate 2 token:
    i. One to use as url query so that browser can go to (probably use uuid.uuid4() library to generate a random token)
    ii. Other in the get request in above link (similarly)
  3. Server send 1st token and generate a site with it and put the 2nd there
  4. Browser request 127.0.0.1:port/?verifyDevice=token
  5. Make the 4 request url Allow all origin header response in server
  6. If Server gets request (that means same device, since 127.0.0.1 can only be used from same device), sends 2nd token, browser gets that
  7. Browser sends the second token to server (same as step 4). Server approves both in browser and server and saves (that token) in user.json and localstorage. Since its in localstorage, unless malicious intent, it won't be shared
  8. Also the request message like "raise volume" will be revarified (using step 4) with the token.

It won't hurt to make some extra requests rather than letting someone control your pc

This is also done in every login or verify user

machine learning

To create a chat AI that can respond to user messages and provide appropriate responses, you will need to follow a few basic steps:

Collect and preprocess data: You will need a dataset of conversation examples to train your chat AI. You can either collect this data manually or use an existing dataset such as Cornell Movie Dialogs Corpus, Ubuntu Dialogue Corpus, etc. After collecting the data, you need to preprocess it to remove noise, normalize text, and convert it into a machine-readable format.

Choose a model architecture: There are various types of models that can be used for chat AI, such as sequence-to-sequence models, transformer models, and memory networks. You can choose the model architecture based on your requirements and the size of your dataset.

Train the model: Once you have chosen a model architecture, you need to train the model on your preprocessed dataset. This involves feeding the model with input-output pairs and adjusting the model's parameters to minimize the loss function.

Test and evaluate the model: After training the model, you need to test it on a separate test dataset to evaluate its performance. You can use metrics such as perplexity, BLEU score, and ROUGE score to evaluate the model's performance.

Deploy the model: Once you are satisfied with the model's performance, you can deploy it to a production environment such as a web application or a chatbot platform.

As for using open source machine learning, there are several libraries and frameworks available that you can use, such as TensorFlow, PyTorch, and Keras. These frameworks provide pre-built models, as well as tools for training and evaluating custom models.

[BUG] Fix convo

Auto catching wrong intent

   Love you >> Love you
   user msg id:  359
   Flags:  {}
   intent: ['say_hello', 'love_you']

hope this helps with bot sending video

You can use various video processing libraries like FFmpeg or GStreamer to manipulate the video files and streaming protocols like HLS or MPEG-DASH to serve them over the internet. You can also use third-party APIs like Cloudinary or Amazon S3 to store and retrieve the video files. However, it's important to note that enabling a chat AI to send videos on a self-hosted website could potentially create legal or ethical issues, especially if the videos are not moderated or monitored for inappropriate content. It's important to carefully consider the implications of such a feature before implementing it.

Sure, here's an example of how you could allow a chat AI to send videos on a self-hosted website using HTML and JavaScript:

First, create an HTML form with an input field for the user to upload a video file:

python

Next, create a JavaScript function to handle the form submission and send the video file to the server using AJAX:

javascript

function uploadVideo() {
var file = document.querySelector('input[type="file"]').files[0];
var xhr = new XMLHttpRequest();
xhr.open('POST', 'upload.php', true);
xhr.upload.onprogress = function(e) {
if (e.lengthComputable) {
var percentComplete = (e.loaded / e.total) * 100;
console.log(percentComplete + '% uploaded');
}
};
xhr.onload = function() {
if (this.status === 200) {
console.log('Video uploaded!');
} else {
console.error('Error uploading video');
}
};
var formData = new FormData();
formData.append('video', file);
xhr.send(formData);
}

Finally, create a PHP script to handle the video upload and store the video file on the server:

php

Note that this code is just an example and would need to be customized to fit your specific website and use case.

Here's an alternative implementation of the same functionality using Node.js and the Express framework:

First, install the necessary dependencies by running the following command in your project directory:

npm install express multer

Next, create a server.js file with the following code:

javascript

const express = require('express');
const multer = require('multer');
const app = express();
const upload = multer({ dest: 'uploads/' });

app.post('/upload', upload.single('video'), (req, res) => {
if (!req.file) {
return res.status(400).send('No video uploaded');
}
res.send('Video uploaded successfully');
});

app.listen(3000, () => {
console.log('Server listening on port 3000');
});

Finally, create an HTML form with an input field for the user to upload a video file, and use JavaScript to submit the form to the server:

php

<script> const form = document.querySelector('form'); form.addEventListener('submit', (e) => { e.preventDefault(); const xhr = new XMLHttpRequest(); xhr.open('POST', '/upload'); xhr.onload = () => { console.log(xhr.responseText); }; const formData = new FormData(form); xhr.send(formData); }); </script>

Again, note that this code is just an example and would need to be customized to fit your specific website and

Asuna Model

Hello, I'm a not a pro python developer so I've got a question. (maybe stupid)

How can I change the model? I don't need a full explanation just some ideas how to do it.

Thanks for your VoiceAI Asuna project. It's amazing!!!!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.