Deep, Narrow Sigmoid Belief Networks Are Universal Approximators

In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impracti...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	Neural computation 2008-11, Vol.20 (11), p.2629-2636
Hauptverfasser:	Sutskever, Ilya, Hinton, Geoffrey E.
Format:	Artikel
Sprache:	eng
Schlagworte:	Accuracy Algorithms Applied sciences Approximation Artificial intelligence Belief networks Biological and medical sciences Computer science control theory systems Exact sciences and technology Fundamental and applied biological sciences. Psychology General aspects Humans Learning Learning and adaptive systems Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects) Miscellaneous Neural Networks (Computer) Nonlinear Dynamics
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impractical way.
ISSN:	0899-7667 1530-888X
DOI:	10.1162/neco.2008.12-07-661