Does finetuning Vocoder on generated mels work? about ims-toucan HOT 3 CLOSED

My implementation is based on this one: https://github.com/kan-bayashi/ParallelWaveGAN

It is uses the same spectrogram representation as in espnet: https://github.com/espnet/espnet

So any vocoder from the espnet model zoo that is trained on 22500Hz should work.

from ims-toucan.

Increasing the sampling rate of the generated spectrograms increases the complexity of the task the model has to solve and its runtime requirements, but yes, with clean data it would probably sound better.

Finetuning the vocoder on TTS outputs is done frequently and is known to bring pretty good improvements for removing artifacts. However, then the vocoder becomes specialized for one certain model and loses some of it's generality. Because I want the vocoder to be universal, I don't plan on doing that and finding other solutions instead, but if you want to try it, it probably increases the quality.

from ims-toucan.

I started working on 22.05K sample rate fs2 model, currently preparing dataset in 22K and will try training new model, but I think the Vocoder will not be compatible since it is trained on 16K datasets, but what other Vocoder do you think will work with the configuration of this module, especially how we handle the Stft part since it is also very important for vocoder along with sample rate?

from ims-toucan.