Natural Gradient Primal-Dual Method for Decentralized Learning

We propose the Natural Gradient Primal-Dual (NGPD) method for decentralized learning of parameters in Deep Neural Networks (DNNs). Conventional approaches, such as the primal-dual method, constrain the local parameters to be similar between connected nodes. However, since most of them follow a first...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	IEEE transactions on signal and information processing over networks 2024, Vol.10, p.417-433
Hauptverfasser:	Niwa, Kenta, Ishii, Hiro, Sawada, Hiroshi, Fujino, Akinori, Harada, Noboru, Yokota, Rio
Format:	Artikel
Sprache:	eng
Schlagworte:	Artificial neural networks Convergence Decentralized learning Gradient methods Image classification Information processing kronecker factored approximate curvature (K-FAC) Machine learning Mathematical analysis natural gradient method Nodes Parameter robustness primal-dual method Resource management Robustness second-order optimization Task analysis
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Schreiben Sie den ersten Kommentar!