Efficient Transformers: A Survey

Transformer model architectures have garnered immense interest lately due to their effectiveness across a range of domains like language, vision and reinforcement learning. In the field of natural language processing for example, Transformers have become an indispensable staple in the modern deep learning stack. Recently, a dizzying number of "X-former" models have been proposed - Reformer, Linfor

[misshiki]

論文“最近、目まぐるしい数の「X-former」モデルが提案されています。いくつか例を挙げると、Reformer、Linformer、Performer、Longformer などがあります。”2020年の論文をアップデートらしい。

[samurairodeo]

“Efficient Transformers: A Survey ”