Curious what causes inconsistent pronunciation of certain words about forwardtacotron HOT 8 CLOSED

At least for Pizza check out:

espeak-ng --ipa -q -x -ven-us "buy data and pizza on the internet"
 bˈaɪ dˈeɪɾə ænd pˈiːtsə ɔnðɪ ˈɪntɚnˌɛt
espeak-ng --ipa -q -x -ven-uk "buy data and pizza on the internet"
 bˈaɪ dˈeɪtə and pˈiːtsəɹ ɒnðɪ ˈɪntənˌɛt

https://github.com/as-ideas/ForwardTacotron/blob/master/hparams.py#L83
"en" defaults to "en-uk", not "en-us"

from forwardtacotron.

Not sure how many such words differ (and in what way) between UK and US English but probably enough to trip it up a bit.

from forwardtacotron.

Hi, I would think that it might be inconsistent in the dataset then. Even with phonemes the model is learning to interpret the input in a ceartain way, although it should be more consistent. If you have two opposing examples you could grep though the dataset and look for similar texts, maybe you see that they are pronounced differently. I did that for weird pronunciations in our German dataset and found some inconsistencies, e.g. the word 'Union' was ambiguous (German and English).

from forwardtacotron.

Hi

Thank you for the information!
As it turns out, can't does not appear at all in my training set! That would explain it for mine.
I will generate some sample sentences, look at failure points then probably add to my training set where there are weaknesses.

But I checked ljspeech and it does have a few occurrences which were all pronounced properly. That said, that may not be enough samples for it to learn what to do when it appears twice in 1 sentence.

from forwardtacotron.

Interesting - it could also be a matter of training longer, I found that it is really easy to judge too quickly, sometimes the model converges after 400k steps only.

from forwardtacotron.

Hi,
I have trained my model to 272K and it still has a lot of the same funny problems. I synthesized the same sentence on the pretained model and it experienced a lot of the same issues:
Here is mine: https://vocaroo.com/eT1Q7T8MQsl
Here is LJSpeech: https://vocaroo.com/1lTbRTS4vhE

The sentence: "Let's walk, and talk, go on the internet, get 100 percent on a test, then use data to buy pizza. After that, we will cause the 1 percent per capita to listen to Mariah, because we are very tired, and want, to go, to sleep..."

It gives this strange Boston accent where words ending in 'a' like data are pronounced dater, capiter, pizzer Mariher, etc.

My dataset is the first 14 chapters of LJSpeech. Maybe it's caused by not enough examples of those types of vowel sounds....

from forwardtacotron.

Oh so I should write en-us! I was not aware of this, I followed the hparams for ljspeech and assumed it was in US English since LJSpeech is US English https://github.com/as-ideas/ForwardTacotron/blob/master/pretrained/pretrained_hparams.py#L81 this would explain why LJSpeech has the same issues.

Thank you I will try to retrain with en-us :)

from forwardtacotron.

Good point @m-toman, I should probably default the LJSpeech hparams to en-us. That would take another week of training though...

from forwardtacotron.