TOTAL MOMENTUM FEEDBACK LOOP FOR BETTER ASGD GENERALIZATION

TOTAL MOMENTUM FEEDBACK LOOP FOR BETTER ASGD GENERALIZATION

A system and a method for distributed training of a machine learning model are disclosed. They are characterized by using feedback loop for total momentum control, and comprising detection of ASGD executions with compromised generalization, a feedback loop for adjusting the total momentum of ASGD ex...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Hauptverfasser: KATZ, Michael, KISILEV, Pavel, TALYANSKY, Roman, MELAMED, Zach
Format: Patent
Sprache:eng ; fre ; ger
Schlagworte:
CALCULATING
COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
COMPUTING
COUNTING
PHYSICS
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:A system and a method for distributed training of a machine learning model are disclosed. They are characterized by using feedback loop for total momentum control, and comprising detection of ASGD executions with compromised generalization, a feedback loop for adjusting the total momentum of ASGD execution, i.e. the sum of the explicit, parametric momentum, and the implicit momentum, related to gradient staleness, toward zero, and tuning ASGD hyperparameters, in accordance with total momentum minimization.