Question: trainer network error about convnetjs HOT 3 OPEN

^ for a similar set up I have the same issue 👍

from convnetjs.

Hello,

I'm back with more questions.
I've implemented the Average Cross-Entropy/Log Loss Error Function for my ideal and prediction values.

1. When running more than one hidden layer, all my network values propagate Nan. Is this intended behavior, loss of precision, or a bug?
2. When using only one hidden layer, I get convergence on absolute values e.g., [1, 0] or [0, 1] but my ACE becomes a NaN.

My implementation is quite simple:

function log_error(actual, ideal)
{
    var err_0 = ideal.w[0] * Math.log(actual.w[0]) + ((1 - ideal.w[0]) * Math.log(1 - actual.w[0]));
    var err_1 = ideal.w[1] * Math.log(actual.w[1]) + ((1 - ideal.w[1]) * Math.log(1 - actual.w[1]));
    return err_0 + err_1;
}

Please note that both ideal and actual are convnetjs.Vol, and I obtain them during training iteration:

var log_sum = 0;
for (var k = 0; k < data.length; k++)
{
    trainer.train(data[k][0], data[k][1]);
    var actual = network.forward(data[k][0]);
    log_sum += log_error(actual, data[k][1]);
}
var ace = -1*(log_sum / data.length);

The variable data is an array which holds a convnetjs.Vol as input data[k][0] and a convnetjs.Vol as output data[k][1] which have been empirically verified.

I understand that the parameters learning_rate play a very important role (I've experimented with very small and large values) as well as the L1 and L2 decay values are important.

I haven't yet cross-validated accuracy, but why do I keep seeing those NaN values?

from convnetjs.

yeah, I am also having NaN issues for a 2-hidden-layer network running deep q learning. Everything is OK for a while and then suddenly I find all my network weights have turned to NaNs.

from convnetjs.