Towards Scalable and Stable Parallelization of Nonlinear RNNs

Conventional nonlinear RNNs are not naturally parallelizable across the sequence length, unlike transformers and linear RNNs. Lim et. al. (2024) therefore tackle parallelized evaluation of nonlinear RNNs, posing it as a fixed point problem solved with Newton's method. By deriving and applying a...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2024-11
Hauptverfasser:	Gonzalez, Xavier, Warrington, Andrew, Smith, Jimmy T H, Linderman, Scott W
Format:	Artikel
Sprache:	eng
Schlagworte:	Algorithms Complexity Newton methods
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!