this line robbed me: mel_spectrogram = F.linear(magnitude.transpose(-1, -2), self.mel_filter_bank)

instead we should use this: torch.bmm(self.mel_filter_bank.expand(input_data.size(0), -1, -1), magnitude).transpose(1, 2)

Hi, dear author, i came across your project by chance, and i am newbie in audio processing, now we already have librosa.stft, i am curious what is the benefit or necessity of using torch-stft, when is it better to use torch-stft , could you please explain it a little, and maybe you can write this in README.md.

Hi!

Thanks for this work!

I'm using this in a model and when I try to export the model to onnx, I get:
Missing key(s) in state_dict: "model_g.dec.stft.forward_basis", "model_g.dec.stft.inverse_basis".

If I change the lines below to persistent=False, the export works... But I'm not sure if this will have any effect that I can't foresee as this persistent flag is not clear to me.

Would you please tell me if this is acceptable?

Thank you!

        self.register_buffer('forward_basis', forward_basis.float(), persistent=False)
        self.register_buffer('inverse_basis', inverse_basis.float(), persistent=False)

Hey, I was wondering if we can port some refactored form of this into pytorch?

After I changed it to DFT, I found that the values of fmin and fmax would affect the magnitude of the amplitude. For example, I know that the frequency of a signal is 156.25Hz and 1562.5Hz, in which FFT=512 and sampling_rate=8000, by calculating that the index where the maximum value and the second largest value of its amplitude are located should be the frequency value. That is, 10 and 100 are calculated correctly by the original DFT, but after the linear calculation, it is found that the calculated value is inconsistent with the actual value, and the result will be affected by fmin and fmax. Why?

I use the exact example code as you provided in the README, but I caught error like this:

Traceback (most recent call last):
  File "my_test.py", line 29, in <module>
    print(np.mean((output - audio) ** 2)) # on order of 1e-16
ValueError: operands could not be broadcast together with shapes (1,1,220416) (1,220500)

I know it's okay to check the shape before further processing, but I just want to know how to resolve that.

Hi,
I've testing your code and it reconstruct your stft.
I'm afraid this is librosa stft

And this is the spectrogram computed with this method

Hi. Is it possible to use your package to apply butterworth filters (or other filters) on timeseries? What would be the syntax?
Thanks so much,
Cat

Hey Prem,
This snippet looks familiar! :)

You don't need to invert the DFT matrix to get the inverse transform. You can do this instead:

# Transform constants
sz,hp = 128,32
wn = hanning( sz+1)[:-1]**.5

# Make DFT matrix, split into real/imag, scale DC/Nyquist to make orthogonal
f  = fft( eye( sz)) / (.5*sqrt( sz) * sqrt( sz/hp))
f = vstack( (real( f[:sz//2+1,:]),imag(f[:sz//2+1,:])))

# This makes the DFT matrix work in both ways using convs
f[0,:] /= sqrt(2)
f[sz//2,:] /= sqrt(2)

# Make the transform kernel
DFT  = torch.FloatTensor( wn * f)[:,None,:]

# Input is bt x dim x time
x = torch.sin( 64 * torch.linspace( 0, 2*pi, 1024, dtype=torch.float32)).view( 1, 1, -1)

# Do DFT
f = F.conv1d( x, DFT, stride=hp, padding=sz)

# Get amplitude and phase
a = torch.sqrt(  f[:,:sz//2+1,:]**2 + f[:,sz//2+1:,:]**2)
p = torch.atan2( f[:,sz//2+1:,:], f[:,:sz//2+1,:])

# ... process ...

# Stack amplitudes and phase
f = torch.cat( [a*torch.cos( p), a*torch.sin( p)], dim=1)

# Inverse STFT using the same kernel
y = F.conv_transpose1d( f, DFT, stride=hp, padding=sz)

Paris

librosa and tensorflow implements have parameter named window_size or frame_size, why there is no such parameter in your implement, thank you!

I want to use this somewhere inside my loss function, it would be great if i was able to get the stft of a batch of signals.

One test has odd behavior: the tolerance for the phase between the librosa STFT and this STFT has to be high for passing. Right now it's set to 1e-2. But the magnitudes match, so something worth looking into there.

I suggest adding some slicing on forward transform in stft.py to keep the output be aligned with the input audio data.

        forward_transform = F.conv1d(
            input_data,
            self.forward_basis,
            stride=self.hop_length,
            padding=0)
        forward_transform = forward_transform[:, :, :-1]

        cutoff = int((self.filter_length / 2) + 1)
        real_part = forward_transform[:, :cutoff, :]
        imag_part = forward_transform[:, cutoff:, :]