Exact gradient updates in time independent of output size for the spherical loss family

An important class of problems involves training deep neural networks with sparse prediction targets of very high dimension D. These occur naturally in e.g. neural language models or the learning of word-embeddings, often posed as predicting the probability of next words among a vocabulary of size D...

Ausführliche Beschreibung

Bibliographische Detailangaben

Veröffentlicht in:	arXiv.org 2016-06
Hauptverfasser:	Vincent, Pascal, de Brébisson, Alexandre, Bouthillier, Xavier
Format:	Artikel
Sprache:	eng
Schlagworte:	Algorithms Back propagation Computation Neural networks Predictions Training Weight
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!