Stability of steepest descent with momentum for quadratic functions

Stability of steepest descent with momentum for quadratic functions

This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. We demonstrate that there always exists a momentum coefficient that will stabilize the steepest descent algorithm, regardless of the value of the learning rate. We also demonstrate how the va...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transaction on neural networks and learning systems 2002-05, Vol.13 (3), p.752-756
Hauptverfasser: Torii, M., Hagan, M.T.
Format: Artikel
Sprache:eng
Schlagworte:
Algorithm design and analysis
Algorithms
Backpropagation algorithms
Coefficients
Convergence
Cost function
Descent
Eigenvalues and eigenfunctions
Equations
Learning
Neural networks
Performance analysis
Stability
Symmetric matrices
Training
Online-Zugang:Volltext bestellen
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. We demonstrate that there always exists a momentum coefficient that will stabilize the steepest descent algorithm, regardless of the value of the learning rate. We also demonstrate how the value of the momentum coefficient changes the convergence properties of the algorithm.
ISSN:1045-9227
2162-237X
1941-0093
2162-2388
DOI:10.1109/TNN.2002.1000143