To understand deep learning we need to understand kernel learning

Generalization performance of classifiers in deep learning has recently become a subject of intense study. Deep models, typically over-parametrized, tend to fit the training data exactly. Despite this "overfitting", they perform well on test data, a phenomenon not yet fully understood. The first point of our paper is that strong performance of overfitted classifiers is not a unique feature of deep

[rjj]

「過学習したモデルがなぜか良い汎化性能を見せる現象は深層学習に特有ではなくカーネル法でも起きる。より簡単なカーネル法でこの現象を解明できなければ深層学習での現象の解明などできないだろう」