Qualitatively characterizing neural network optimization problems

Training neural networks involves solving large-scale non-convex optimization problems. This task has long been believed to be extremely difficult, with fear of local minima and other obstacles motivating a variety of schemes to improve optimization, such as unsupervised pretraining. However, modern neural networks are able to achieve negligible training error on complex tasks, using only direct t

[yu4u]

そういえばICLR2017読み会で出てた、flat minimum / sharp minimumの論文でやってた、2つのNNのパラメータを加重平均して、加重パラメータを動かしながらロスを見るの、これが初出なのか。