Deformed semicircle law and concentration of nonlinear random matrices for ultra-wide neural networks

In this paper, we investigate a two-layer fully connected neural network of the form f (X) = 1/√d1 a⊤σ (W X), where X ∈ d0 × n is a deterministic data matrix, W ∈ Rd1 × d0 and a ∈ Rd1 are random Gaussian weights, and σ is a nonlinear activation function. We study the limiting spectral distributions...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	The Annals of applied probability 2024-04, Vol.34 (2), p.1896
Hauptverfasser:	Wang, Zhichao, Zhu, Yizhe
Format:	Artikel
Sprache:	eng
Schlagworte:	Asymptotic properties Constraining Covariance matrix Deformation Eigenvalues Lower bounds Matrix Neural networks Regression
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!