Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

We introduce the "exponential linear unit" (ELU) which speeds up learning in deep neural networks and leads to higher classification accuracies. Like rectified linear units (ReLUs), leaky ReLUs (LReLUs) and parametrized ReLUs (PReLUs), ELUs alleviate the vanishing gradient problem via the identity for positive values. However, ELUs have improved learning characteristics compared to the units with

[stealthinu]

ReLU/LeakyReLU/ELU/SReLUの比較がある。結果見るとSReLUが単純でELUと最終的な結果がだいたい同じになってるから一番いいような感じ？なぜこの関数だと良い結果出るのかはわからぬが。