Variance Reduction in Deep Learning: More Momentum is All You Need

Variance reduction (VR) techniques have contributed significantly to accelerating learning with massive datasets in the smooth and strongly convex setting (Schmidt et al., 2017; Johnson & Zhang, 2013; Roux et al., 2012). However, such techniques have not yet met the same success in the realm of...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2021-11
Hauptverfasser:	Tondji, Lionel, Kashubin, Sergii, Cisse, Moustapha
Format:	Artikel
Sprache:	eng
Schlagworte:	Clustering Datasets Deep learning Massive data points Momentum Optimization Regularization Regularization methods
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!