For "unseen Speaker Adaptation" , did you refine the model using the data of target speaker, or just using MelStyleEncoder to get ws which adjust the output (like zero-shot/one-shot)?

I have trained the model for 200k steps, and still, the synthesised results are extremely bad. The sampling rate I have used is 22050 Hz and the batch size used is 16.

This is how my loss curve looks after 200k steps. Can you help me with what can I do now to improve my synthesized audio results?

After running the inference command, the result is a png file showing spectrograms. So I tried to do some changes in the synthesize.py file to covert generated mel -spectrogram to audio file. I used librosa for this purpose. Below is the code snippet to do so.

But the audio obtained is blank.
Can you please help me out to convert the mel to proper audio? I am not yet an expert in this field. @KevinMIN95 your help is appreciated here.
Thank you

How do you train the vocoder? Have you used the GTA training?

Hello, please check and help me fix it.

python preprocess.py -> AttributeError: module 'preprocessors.libritts' has no attribute 'write_metadata'

I use two speakers， but the result is bad！

Hi. I notice that in the demo page (Section 4.3), you only did parallel voice cloning with unseen speakers, have you tried testing with different text with these unseen speakers?
Besides, is it possible to get some pretrained models?
Thank you very much.

@KevinMIN95 I use this pretrained model: https://huggingface.co/Guan-Ting/StyleSpeech-MelGAN-vocoder-16kHz. And I also use this repos https://github.com/descriptinc/melgan-neurips for inference, but the output audio sound very bad, output pitch may changed.
1463_infer.zip

@KevinMIN95 Could you please tag the author of this model if it necessary.

When running train.py,
models/VarianceAdaptor.py line 52: x = self.ln(x) + pitch_embedding + energy_embedding

returns an error.
The shape of x seems to be [Batch_size, max_text_input_length, 256].
The shape of the other two seems to be [Batch_size, ??????, 256] (I don't know what pitch_embedding.shape(1) should be.)

Is there a solution to this?

Dear author @KevinMIN95

Thank you for sharing the interesting project.
I use pretrained model (Stylespeech and Meta-Stylespeech) and Melgan (pretrained model).
I also use the same people in the page (https://stylespeech.github.io/) to evaluate Trained Speaker and Unseen Speaker. However, the synthesized audio is not good as you report.
Could you give me some advice to reproduce the similar result that you report.
Best Regard

As I understand, you train your own version of MelGAN for multi-speaker synthesis, as the official code supports the sampling rate of 22.05 kHz, while StyleSpeech operates at 16 kHz.
Could you share the details for reproducibility purposes: which dataset did you use, which parameters did you change? Or you can maybe upload the trained vocoder itself? It would be great!

When I ran the preprocess.py, I would get this error: IndexError: cannot do a non-empty take from an empty axes.

Traceback (most recent call last):
File "preprocess.py", line 54, in
main(preprocessor, args.data_path, args.output_path)
File "preprocess.py", line 39, in main
datas = preprocessor.build_from_path(data_dir, out_dir)
File "/media/fish-bsp/fish_4TB/Audio/StyleSpeech-main/preprocessors/libritts.py", line 132, in build_from_path
f0 = remove_outlier(f0)

I think the problem is from here:

How can I revise the code to let it run successfully? Thanks.

Same as the title, I use the HiFi-GAN vocoder to generate the audio. But there is full of noice in the audio. How could you make the qualified audio as the demo page. Could you pls share some experinece.

Thanks a lot.

Do you have chinese-data model?

I have trained a stylespeech model use LibriTTS, but the quality was far worse than pretrain stylespeech model of author. I use default config and parameter and train the model within 100k step. The loss like bellow:

I also upload my audio sample of the text same as demo page in folder Train_LibriTTS_StyleSpeech in attached fille. There are always strange sounds at the end of each audio file, i can't explain that.
meta_stylespeech_results.zip

Hi,
I'm probably missing something, but how can I generate the synthesized audio file from the mel-spectrogram in the synthesize.py demo?
Thanks

@KevinMIN95 Why you use model_without_ddp and discriminator_without_ddp to calculate some tensors participating the losses calculation? I think the gradients of model_without_ddp will not be synchronized and reduced accross the device, and could this lead to mistakes in distributed training?

I have tried to train a lot of model with sampling rate 22050 but, it can not reproduce quality of 16000 hz model. Can you explain why you use 16000 in your research ?