batch size > 1 for varying sequence lengths about wtte-rnn HOT 8 CLOSED

@daynebatten I actually just noticed that you use model.add(Masking(mask_value=0., input_shape=(max_time, 24))) link

But it was there all along and sorry, actually it shouldn't be a problem for the Turbofan dataset since sensors measure . The problem would occur in data [data_pipeline}(https://github.com/ragulpr/wtte-rnn/blob/master/examples/data_pipeline/data_pipeline.ipynb)-example where we have long periods of nothingness, but we still want to propagate state. Here data could be set to 0. If we'd have mask_value=0 we wouldn't predict or learn anything here.

If you want to apply to other dataset I suggest using some highly unlikely mask-value instead.

Also, we never said hi awesome work @daynebatten! 👏🙌

from wtte-rnn.

That's a great point. It's probably a best practice to use a very large mask value. If you've normalized your data, the mask value should never come up by accident, and certainly not for all variables.

And I think most of the thanks go to you for doing all the heavy lifting here!

from wtte-rnn.

@Manelmc Good catch. While one would expect masking to propagate to the loss function, there was some inconsistencies in Keras at the time of implementation w.r.t how _weighted_masked_objective worked back then with custom loss function. It might have been fixed now. If that's the case, then sample_weights are not necessary if you want equal weights.

I like to use sample_weights anyway since I usually don't use equal weights for every sample.

from wtte-rnn.

I'm with you. Batch_size >1 is inshallah coming tomorrow in the updated data_pipeline. If you can't wait, the tests gives you a hint:
https://github.com/ragulpr/wtte-rnn/blob/master/python/tests/test_keras.py#L93

There's allready support for it. You need to do masking and use sample weights 👌

from wtte-rnn.

thanks! think I got it.

from wtte-rnn.

Note that if you use zero-mask then you may not learn anything from the nonevents as the input vec may be zero for those steps. Ping @daynebatten

from wtte-rnn.

Not sure I'm following, @ragulpr. Can you give a little more detail as to when this would occur?

from wtte-rnn.

Hi Egil, I'm not sure I understand why do you suggest to use sample weights if we are already masking. From what I read in stackoverflow

If there's a mask in your model, it'll be propagated layer-by-layer and eventually applied to the loss. So if you're padding and masking the sequences in a correct way, the loss on the padding placeholders would be ignored.

If the loss on the padding placeholders is ignored, why do we need sample weights?

from wtte-rnn.