How to use dynamic phonation feature with mfcc/fbank feature as the input to feed a DNN. about disvoice HOT 2 OPEN

Hi @ASR2020Guru

You are right on the fact that a zero padding is not the right way to combine those features

There are two reasons why you are getting different lengths for the phonation and fbank features

1. Phonation features are only computed for speech segments where there is F0 values, i.e., only for voiced segments

Check the code in

for l in range(nF):
    data_frame=data_audio[int(l*size_stepS):int(l*size_stepS+size_frameS)]
    energy=10*logEnergy(data_frame)
    if F0[l]!=0:
        Amp.append(np.max(np.abs(data_frame)))
        logE.append(energy)
        if lnz>=12: 
            amp_arr=np.asarray([Amp[j] for j in range(lnz-12, lnz)])
            #print(amp_arr)
            apq.append(APQ(amp_arr))
        if lnz>=6: # TODO:
            f0arr=np.asarray([F0nz[j] for j in range(lnz-6, lnz)])
            ppq.append(PPQ(1/f0arr))
        lnz=lnz+1

In case you want to combine the features you should add an else: statement and add zero values to variables Amp, logE, apq, and ppq

2. In addition, you should consider that apq is only computed after the 12th frame because it is a log-term perturbation with respect to the 11th previous frames, thus they have to padd 11 zeros at the beginning for this feature.

3. The same ocurrs for ppq, but in this case with the first five frames

If you add these padds at the beginning for apq and ppq you should remove this line where it considers only those frames after the 12th, in orderto properly merge apq and ppq with the rest of the features

If you have further questions, let me know and I can help you

from disvoice.

Hi @jcvasquezc ,

Thanks for your quick and helpful reply.

Now I managed to combine these features correctly.

I will let you know if I have any further questions.

Cheers

from disvoice.