A Convergence Theory for Deep Learning via Over-Parameterization

Deep neural networks (DNNs) have demonstrated dominating performance in many fields; since AlexNet, networks used in practice are going wider and deeper. On the theoretical side, a long line of works has been focusing on training neural networks with one hidden layer. The theory of multi-layer networks remains largely unsettled. In this work, we prove why stochastic gradient descent (SGD) can find

[stealthinu]

ディープニューラルネットで（ReLUとSGDで）パラメータ数が十分なら最適解に収束すると数学的に証明されたらしい。全結合もCNNもResNetにも適用可能な理論とのこと。

[hhooggee]

テストデータに対して100%の精度が出るって過学習では

[Nyoho]

まじかよすごいじゃん