A new paper by Liu, Jian, He et al introduces RAdam, or “Rectified Adam”. It’s a new variation of the classic Adam optimizer that provides an automated, dynamic adjustment to the adaptive learning rate based on their detailed study into the effects of variance and momentum during training. RAdam holds the promise of immediately improving every AI architecture compared to vanilla Adam as a result:
![New State of the Art AI Optimizer: Rectified Adam (RAdam).](https://cdn-ak-scissors.b.st-hatena.com/image/square/4a34ef22cbf970317d0ba6cd09950fd9ce588aef/height=288;version=1;width=512/https%3A%2F%2Fmiro.medium.com%2Fv2%2Fresize%3Afit%3A1200%2F1%2ABMwu8Km-CtPsvaH8OM5_-g.jpeg)