Hello,
Is there any "magic trick/formula" to guarantee that the lyrics will always match the midi file when you play them together? My resulting output sounds good to the ear and nothing seems out of place until I add the midi instrumentals in the background, the timing is off even if the output was generated using the same tempo or the two files have the same duration, even when after some tweaking the lyrics are pronounced with the correct timing, the duration of the entire file is off, so I have to manually cut and paste the lyrics from an audio editor and glue them in the correct place, which is quite a tedious work. It would be great if it was possible to play the resulting output and midi file together right out of the box, or understand the logic why this problem happens.
Thanks for the amazing repo, I'm really enjoying this.

Hello mathigatti, thank you alot for this. I am excited to get it to work but i am having an error when i try and make the Sinsy file on linux (Linux Mint 19.3). This is the last bit of the error message i get:

[100%] Linking CXX executable sinsyNG
CMakeFiles/sinsyNG.dir/bin/sinsy.o: In function ECantorix::init(double)': sinsy.cpp:(.text._ZN9ECantorix4initEd[_ZN9ECantorix4initEd]+0x8d): undefined reference to sinsy_ng_Init'
CMakeFiles/sinsyNG.dir/bin/sinsy.o: In function ECantorix::addNote(unsigned long, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, unsigned long, bool, bool, unsigned long, unsigned long, unsigned long, bool)': sinsy.cpp:(.text._ZN9ECantorix7addNoteEmRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmbbmmmb[_ZN9ECantorix7addNoteEmRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmbbmmmb]+0xd0): undefined reference to sinsy_ng_addNote'
sinsy.cpp:(.text._ZN9ECantorix7addNoteEmRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmbbmmmb[_ZN9ECantorix7addNoteEmRKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEmbbmmmb]+0x12a): undefined reference to sinsy_ng_addNote' CMakeFiles/sinsyNG.dir/bin/sinsy.o: In function ECantorix::addRest(unsigned long)':
sinsy.cpp:(.text._ZN9ECantorix7addRestEm[_ZN9ECantorix7addRestEm]+0x4e): undefined reference to sinsy_ng_addRest' CMakeFiles/sinsyNG.dir/bin/sinsy.o: In function ECantorix::saveTo(std::__cxx11::basic_string<char, std::char_traits, std::allocator >)':
sinsy.cpp:(.text._ZN9ECantorix6saveToENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE[_ZN9ECantorix6saveToENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE]+0x3a): undefined reference to `sinsy_ng_getAudioData'
collect2: error: ld returned 1 exit status
CMakeFiles/sinsyNG.dir/build.make:95: recipe for target 'sinsyNG' failed
make[2]: *** [sinsyNG] Error 1

Maybe you can help, thank u!

Hello,
Sorry for being a total python noob, to my understanding, main.py creates 3 files on /tmp ("last_midi_generated_by_vrender.mid", "last_voice.musicxml" and "last_voice.xml") with my lyrics on repeat to the 4 notes that are hardcoded, then outputs a "output.wav" file, however, when trying to define a tempo and custom output name (as mentioned in the README.md file) doesn't seem to have any effect at all.
Can you show me how to have a fully working main.py please ? The idea is to use a command-line such as "python main.py singing_output.wav "some lyrics to sing" 100 song.mid".
Until now I managed to do it manually with "musescore ./tmp/song.mid -o ./tmp/last_voice.musicxml", run main.py but preventing musescore to be found so it won't not generate a "last_midi_generated_by_vrender.mid" on ./tmp but will generate a valid "last_voice.xml" from the "last_voice.musicxml" I created before, then load "sinsyNG -m en -o output.wav ./tmp/last_voice.xml", however, the output duration is much longer than the actual midi (ie. output.wav is 6 minutes vs song.mid is 3 minutes), I couldn't figure out how to manually change the tempo so the output.wav lyrics matches with the midi instruments.
Any help is much appreciated, thank you for your kind attention!

Colab Error

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-14-0a7be42dee8f>](https://localhost:8080/#) in <cell line: 3>()
      1 # Press the play button on this cell for loading the new result
      2 # Then the new audio file will be available to listen in the player
----> 3 ipd.Audio("/content/RealTimeSingingSynthesizer/output.wav")

[/usr/local/lib/python3.10/dist-packages/IPython/lib/display.py](https://localhost:8080/#) in __init__(self, data, filename, url, embed, rate, autoplay, normalize, element_id)
    114         if self.data is not None and not isinstance(self.data, bytes):
    115             if rate is None:
--> 116                 raise ValueError("rate must be specified when data is a numpy array or list of audio samples.")
    117             self.data = Audio._make_wav(data, rate, normalize)
    118 

ValueError: rate must be specified when data is a numpy array or list of audio samples.

secondly there is no output file.

Hi,

Is there a way to use MBROLA voices?
I have tried to test this in Colab (installed mbrola and the mbrola voices as well), but the result is empty (the length of the output looks fine, but there is just silence).

Another question:
Do you have plan to make the notes of the singing connected to the syllables instead of the words?

Thanks!