Applying deep learning to translate animation and re-generate audio.

Supported output languages:

Click to view Demo video (Japanese to Chinese) on YouTube

Input video and translate it to Chinese (Mandarin):

python3 convert.py ~/Desktop/test.mp4 --target_language Chinese

Input video, subtitle file (*.srt) and translate it to Chinese (Mandarin):

python3 convert.py ~/Desktop/test.mp4 --srt ~/Desktop/test.srt --target_language Chinese

⚠️ Parsing SRT file is on Todo list. This feature is not available.

git clone https://github.com/NewJerseyStyle/anime-translator.git
cd anime-translator
pip install -r requirements.txt
python3 convert.py ~/Desktop/test.mp4

Credits to powerful tools made this repo possible:

• MoviePy
• whisperX project
• OpenAI Whisper
• Microsoft VALL-E (X)
• PyDub
• librosa
• Facebook Demucs
• translatepy

• it tries to sing during OP/ED