Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covar

[elu_18]

今度はGoogleがILSVRC2012分類タスクのテストエラーを4.82%に更新 各層ごとに活性をミニバッチ単位で正規化することで、いろんな正則化テクニックがいらなくなる上に学習が14倍速くなる http://t.co/d8rlUnH1WC

[peroon]

BNは精度ではなく訓練速度を上げる

[fcicq]

see also 1603.05027

[rishida]

バッチ処理で学習が早くなる話