Error in LRN about miopen HOT 6 CLOSED

Hi,
Could you be more specific, please?
What configuration is causing the problem?
Or have you found an obvious bug in the source code?
Thank you a lot, we enormously appreciate your feedback.
Best regards,
MIOpen team.

from miopen.

Hi!
I meant that in cuDNN LRN is calculated like:

y_ij = |x_ij| / (K + alpha * (sum_kl x_kl^2) / N^2)^beta

and in MIOpen:

y_ij = x_ij / (K + alpha * (sum_kl x_kl^2) / N^2)^beta

(without abs value of x_ij)

So, may be it should be changed, so in deep learning frameworks LRN layers for NVIDIA and AMD would be similar?

from miopen.

Here is, roughly, the cuda code:
template
global void LRNComputeOutput(const int nthreads, const Dtype* const in,
const Dtype* const scale, const Dtype negative_beta, Dtype* const out) {
CUDA_KERNEL_LOOP(index, nthreads) {
out[index] = in[index] * pow(scale[index], negative_beta);
}
}

X is in[index]
Y is out[index]

I’m not sure where is the difference?

I do not see any abs function here.
Abs will indeed cause the sign loss but LRN is a scaling operation it should keep sign unchanged.

Best regards,
MIOpen team.

from miopen.

Sorry, I meant that in cuDNN it is without abs.
If you look at MIOpen LRN line 385, there is bot_val * s. And bot_val (line 381) is sqrt of something. Therefore, it is always >= 0, we lost a sign.

from miopen.

you're right. some minor optimization can cause a lot of havoc :)
thank you, great catch. we've fixed it already and put it into the next release.
Best regards,
MIOpen team.

from miopen.

@ashmanov this issue has been fixed in the latest release.

from miopen.