sapphirine / 202305_17_lipsync_translation Goto Github PK

202305_17_LipSync_Translation

Automated Video Translation and Content Generation using Talking Heads, Leveraging Wav2Lip Model

Ethical Use Disclaimer :

This repo can only be used for research/academic/personal purposes as it internally uses wav2lip model which restricts any commercial usage.
Please refere here for further license and commerical usage : https://github.com/Rudrabha/Wav2Lip

Deployment doc has been provided in the repo & can also be found in below url for the most recent changes

Deployment information provided here :
https://docs.google.com/document/d/1WQfjUoaR4CzYFrDXdm5scRqK0TLLoExy5ihs5hNp7cQ/edit

Deployment

https://github.com/Sapphirine/202305_17_LipSync_Translation/tree/main/bigdataproject/setup

Architecture : Original

Architecture : Kafka

Wav2Lip Paper Citation

@inproceedings{10.1145/3394171.3413532, author = {Prajwal, K R and Mukhopadhyay, Rudrabha and Namboodiri, Vinay P. and Jawahar, C.V.}, title = {A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild}, year = {2020}, isbn = {9781450379885}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3394171.3413532}, doi = {10.1145/3394171.3413532}, booktitle = {Proceedings of the 28th ACM International Conference on Multimedia}, pages = {484–492}, numpages = {9}, keywords = {lip sync, talking face generation, video generation}, location = {Seattle, WA, USA}, series = {MM '20} }