the value of the residual vector quantization commitment loss about encodec HOT 3 OPEN

Good catch, we forgot to document it. The value is 1. Because the commitment loss is not defined with respect to the output of the model, it is not included in the balancer, and its gradient is computed separately and added to the one obtained from the balancer.

from encodec.

Hi, @adefossez how to add the gradient of vector quantization commitment loss into the balancer? I try to use backpropagation to calculate the gradient value of vector quantization commitment loss separately, and then use the balancer to calculate the gradient of the remaining losses and update the network parameters as follows：
loss_vq.backward(retain_graph = True)
balancer.backward(loss_g,x_hat)
But I do not get good results. Could you please share some methods to add this gradient to balancer?

from encodec.

We do not add this to the balancer, the loss of the commitment loss is added during a second backward.

from encodec.