ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training Published April 19, 2021 By DeepSpeed Team Rangan Majumder , Vice President Andrey Proskurin , Corporate Vice President of Engineering Since the DeepSpeed optimization library was introduced last year, it has rolled out numerous novel optimizations for training large AI models—improving scale, speed, cost,
![ZeRO-Infinity and DeepSpeed: Unlocking unprecedented model scale for deep learning training - Microsoft Research](https://cdn-ak-scissors.b.st-hatena.com/image/square/ada0e2195ab17e4b7a92af0beb0e4de4492cb6d2/height=288;version=1;width=512/https%3A%2F%2Fwww.microsoft.com%2Fen-us%2Fresearch%2Fuploads%2Fprod%2F2020%2F05%2F1400x788_deepspeed_update_figure_nologo_Still-2_04-2020-scaled.jpg)