Stochastic Polyak Step-sizes and Momentum: Convergence Guarantees and Practical Performance

Stochastic gradient descent with momentum, also known as Stochastic Heavy Ball method (SHB), is one of the most popular algorithms for solving large-scale stochastic optimization problems in various machine learning tasks. In practical scenarios, tuning the step-size and momentum parameters of the m...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Oikonomou, Dimitris, Loizou, Nicolas
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Learning Mathematics - Optimization and Control Statistics - Machine Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!