Entropic gradient descent algorithms and wide flat minima

The properties of flat minima in the empirical risk landscape of neural networks have been debated for some time. Increasing evidence suggests they possess better generalization capabilities with respect to sharp ones. First, we discuss Gaussian mixture classification models and show analytically th...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2021-11
Hauptverfasser:	Pittorino, Fabrizio, Lucibello, Carlo, Feinauer, Christoph, Perugini, Gabriele, Baldassi, Carlo, Demyanenko, Elizaveta, Zecchina, Riccardo
Format:	Artikel
Sprache:	eng
Schlagworte:	Algorithms Computer Science - Learning Correlation analysis Empirical analysis Entropy Estimators Flatness Machine learning Minima Neural networks Optimization Physics - Disordered Systems and Neural Networks Statistics - Machine Learning
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!