Denoising Autoencoders target label about handson-unsupervised-learning HOT 5 CLOSED

Hi, here I am simulating a case where we do not have the clean dataset at all. All we have to work with is the "noisy" data. The goal of the this autoencoder is to ignore the noise and instead learn to reconstruct just the core elements of X_train_AE. We don't want the autoencoder to learn to reconstruct perfectly X_train_AE_noisy at all. We want the autoencoder to learn to ignore the noise while it's learning. That's why we train on X_train_AE_noisy but we evaluate on X_train_AE. Please drop me a note if this is still unclear.

from handson-unsupervised-learning.

Great question. By underfitting the autoencoder on the noisy data, the autoencoder learns the common "signal" but ignores the rare "noise". If the noise truly drowns out the signal in an overwhelming way, even underfitting the autoencoder won't cut it. But, in instances where the noise is present but not the majority of the data, underfitting works well in practice.

Does this underfitting clarify how using an autoencoder on a noisy dataset still achieves the end objective of learning the signal in the data but ignoring the noise?

from handson-unsupervised-learning.

from handson-unsupervised-learning.

Thank you for the reply,

The goal of the this autoencoder is to ignore the noise and instead learn to reconstruct just the core elements of X_train_AE.

The concept is clear, but what is not clear to me is how we get this if the objective of the learning (the target of training) is the noisy input Itselfs. How do we "teach" to the Autoencoder what is the noise if It only sees noised data ?

from handson-unsupervised-learning.

Yeah thank you! I have a more clear idea now.

Anyway, what about training the Autoencoder to reconstruct the original data (without noise) ? Could be an effective alternative ? (Just to complete my overview on this topic)
Obiously with the constraint that we have the un-noised data.

from handson-unsupervised-learning.