esterhlav / dynamical-isometry-from-orthogonality-neural-nets Goto Github PK
View Code? Open in Web Editor NEWMathematical consequences of orthogonal weights initialization and regularization in deep learning. Experiments with gain-adjusted orthogonal regularizer on RNNs with SeqMNIST dataset.