A BF16 FMA is All You Need for DNN Training

Fused Multiply-Add (FMA) functional units constitute a fundamental hardware component to train Deep Neural Networks (DNNs). Its silicon area grows quadratically with the mantissa bit count of the computer number format, which has motivated the adoption of the BrainFloat16 format (BF16). BF16 feature...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on emerging topics in computing 2022-07, Vol.10 (3), p.1302-1314
Hauptverfasser:	Osorio, John, Armejach, Adria, Petit, Eric, Henry, Greg, Casas, Marc
Format:	Artikel
Sprache:	eng
Schlagworte:	Accuracy Arithmetic Artificial neural networks BF16 Compounds Computational modeling computer arithmetic emulation Floating point arithmetic FMA Operators Format FP32 Hardware machine learning Neural nets Operators Proposals reduced precision Silicon swamping Training
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Schreiben Sie den ersten Kommentar!