Hope to get your answer. Thank you! about self-ensemble-visual-domain-adapt HOT 3 CLOSED

1. Although both networks are initialised randomly, the EMAWeightOptimizer constructor starts by first copying the student weights over those of the teacher (lines 29-30), so the teacher and student will be identical before training starts.
2. Not really. The teacher network should follow the student slowly, hence teacher = teacher * 0.99 + student * 0.01. The 'trajectory' of its weights is therefore more smooth. It can therefore be used in effect to produce reliable pseudo-labels.
3. It erases the weights of the teacher by copying the weights of the student over them.

from self-ensemble-visual-domain-adapt.

If the two weights are the same before training,
How to train the consistency loss? Isn't the goal of training consistency loss to make the weight of the two net equal?
When you execute teacher = teacher * 0.99 + student * 0.01, the teacher weight cannot change. Isn't it?

from self-ensemble-visual-domain-adapt.

Please read the 'temporal ensembling' (https://arxiv.org/abs/1610.02242) and 'mean teachers are better role models' (https://arxiv.org/abs/1703.01780) papers; they should give you a better idea of how consistency regularization and the mean teacher model work.

from self-ensemble-visual-domain-adapt.