Train With Mixed Precision - NVIDIA Docs

Abstract Mixed precision methods combine the use of different numerical formats in one computational workload. This document describes the application of mixed precision to deep neural network training. There are numerous benefits to using numerical formats with lower precision than 32-bit floating point. First, they require less memory, enabling the training and deployment of larger neural networ

[rawwell]

“A and B are half precision 4x4 matrices, whereas D and C can be either half or single precision 4x4 matrices”