Attention neural networks with sparse attention mechanisms

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing network inputs using an attention neural network that has one or more sparse attention sub-layers. Each sparse attention sub-layer is configured to apply a sparse attention mechanism that a...

Ausführliche Beschreibung

Bibliographische Detailangaben

Hauptverfasser:	Guruganesh, Guru, Ahmed, Amr, Pham, Philip, Ainslie, Joshua Timothy, Ontañón, Santiago, Zaheer, Manzil, Dubey, Kumar Avinava
Format:	Patent
Sprache:	eng
Schlagworte:	CALCULATING COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS COMPUTING COUNTING PHYSICS
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!