Train in one language and synthesize in English about styletts2 HOT 3 CLOSED

Hi -- yes, you can. Typically when I train StyleTTS2 to synthesise another language, I keep a bit of English in the dataset, just to make sure I retain the English speaking capability.

The other thing is that the reference you use often matters. If you are using a reference from a (for example) Spanish speaker and try to synthesise English, then you might run into an issue where you get a Spanish accent.

If none of this works, you can try to use this multilingual PL-BERT model on Huggingface - I just open-sourced it. https://huggingface.co/papercup-ai/multilingual-pl-bert

from styletts2.

You can maybe use RVC (or any voice conversion model) to convert an english reference to the target speaker you are trying to synthesise, and then use the RVC output as the reference for styletts2

from styletts2.

Got it, thanks for your reply! I was imagining that mixing the languages might be a solution.

I didn't pay attention to the reference, though. I have this problem that I don't have the reference in English, so maybe I'll think how can I get to work in this case. Do you have any ideas?

Also, good work with the PL Bert. It will definitely help me and others!

from styletts2.