GitHub - NVIDIA/Megatron-LM: Ongoing research training transformer models at scale

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session. Dismiss alert

[rawwell]

“we developed a simple and efficient two-dimensional model-parallel approach. To use tensor model parallelism (splitting execution of a single transformer module over multiple GPUs), add the --tensor-model-parallel-size flag to specify the number of GPUs among which to split the model, along with

[hnishi2509]

これ動かせる環境持っている人ってよっぽどやなぁ。“We have provided an example of how to configure Megatron to run GPT-3 with 175 billion parameters on 1024 GPUs.”