Adam [1] is an adaptive learning rate optimization algorithm that’s been designed specifically for training deep neural networks. First published in 2014, Adam was presented at a very prestigious conference for deep learning practitioners — ICLR 2015. The paper contained some very promising diagrams, showing huge performance gains in terms of speed of training. However, after a while people starte
![Adam — latest trends in deep learning optimization.](https://cdn-ak-scissors.b.st-hatena.com/image/square/88627393c13c83ebf7c8970eaa1b0c6ba68076ce/height=288;version=1;width=512/https%3A%2F%2Fmiro.medium.com%2Fv2%2Fresize%3Afit%3A1200%2F1%2AlAMbp2AXAVlmxN_hbsDi3g.jpeg)