Deep, Narrow Sigmoid Belief Networks Are Universal Approximators

In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impracti...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	Neural computation 2008-11, Vol.20 (11), p.2629-2636
Hauptverfasser:	Sutskever, Ilya, Hinton, Geoffrey E.
Format:	Artikel
Sprache:	eng
Schlagworte:	Accuracy Algorithms Applied sciences Approximation Artificial intelligence Belief networks Biological and medical sciences Computer science control theory systems Exact sciences and technology Fundamental and applied biological sciences. Psychology General aspects Humans Learning Learning and adaptive systems Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects) Miscellaneous Neural Networks (Computer) Nonlinear Dynamics
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

container_end_page	2636
container_issue	11
container_start_page	2629
container_title	Neural computation
container_volume	20
creator	Sutskever, Ilya Hinton, Geoffrey E.
description	In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impractical way.
doi_str_mv	10.1162/neco.2008.12-07-661
format	Article
fullrecord	<record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmed_primary_18533819</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>21043336</sourcerecordid><originalsourceid>FETCH-LOGICAL-c638t-a0777d1dd63fa9d9e5d13d0e5470e2cdab3f4cc890780015c714ea8bbdf6a9fc3</originalsourceid><addsrcrecordid>eNqNkUtvFDEQhC1ERJbAL0BCIyQ4ZTbdtsf23FhCHkhROEAkbpbX9iCHmfFg7ybAr49XuwKEeJ368lV1dRchTxDmiIIejd7GOQVQc6Q1yFoIvEdm2DColVIf7pMZqLatpRBynzzM-RoABELzgOyjahhT2M7Iy9feT4fVpUkp3lbvwschBle98n3wXXXpV7cxfcrVIvnqagw3PmXTV4tpSvFLGMwqpvyI7HWmz_7xbh6Qq9OT98fn9cXbszfHi4vaCqZWtQEppUPnBOtM61rfOGQOfMMleGqdWbKOW6takAoAGyuRe6OWS9cJ03aWHZAXW9-y-_Pa55UeQra-783o4zpr0QrkwNk_QYogqOLN_4DFjokCPvsFvI7rNJZrC4MUleK8QGwL2RRzTr7TUyovSl81gt70pTd96U1fGqkGqUtfRfV0Z71eDt790OwKKsDzHWCyNX2XzGhD_s5RkKJBUIU733JD-CneZuUNhYCoGVDeyCKgWOLokuJbmP6Q6eg3Vn-74g6FkMK8</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>211218844</pqid></control><display><type>article</type><title>Deep, Narrow Sigmoid Belief Networks Are Universal Approximators</title><source>MEDLINE</source><source>MIT Press Journals</source><creator>Sutskever, Ilya ; Hinton, Geoffrey E.</creator><creatorcontrib>Sutskever, Ilya ; Hinton, Geoffrey E.</creatorcontrib><description>In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impractical way.</description><identifier>ISSN: 0899-7667</identifier><identifier>EISSN: 1530-888X</identifier><identifier>DOI: 10.1162/neco.2008.12-07-661</identifier><identifier>PMID: 18533819</identifier><identifier>CODEN: NEUCEB</identifier><language>eng</language><publisher>One Rogers Street, Cambridge, MA 02142-1209, USA: MIT Press</publisher><subject>Accuracy ; Algorithms ; Applied sciences ; Approximation ; Artificial intelligence ; Belief networks ; Biological and medical sciences ; Computer science; control theory; systems ; Exact sciences and technology ; Fundamental and applied biological sciences. Psychology ; General aspects ; Humans ; Learning ; Learning and adaptive systems ; Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects) ; Miscellaneous ; Neural Networks (Computer) ; Nonlinear Dynamics</subject><ispartof>Neural computation, 2008-11, Vol.20 (11), p.2629-2636</ispartof><rights>2008 INIST-CNRS</rights><rights>Copyright MIT Press Journals Nov 2008</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c638t-a0777d1dd63fa9d9e5d13d0e5470e2cdab3f4cc890780015c714ea8bbdf6a9fc3</citedby><cites>FETCH-LOGICAL-c638t-a0777d1dd63fa9d9e5d13d0e5470e2cdab3f4cc890780015c714ea8bbdf6a9fc3</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://direct.mit.edu/neco/article/doi/10.1162/neco.2008.12-07-661$$EHTML$$P50$$Gmit$$H</linktohtml><link.rule.ids>314,780,784,27924,27925,54009,54010</link.rule.ids><backlink>$$Uhttp://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=20765108$$DView record in Pascal Francis$$Hfree_for_read</backlink><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/18533819$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Sutskever, Ilya</creatorcontrib><creatorcontrib>Hinton, Geoffrey E.</creatorcontrib><title>Deep, Narrow Sigmoid Belief Networks Are Universal Approximators</title><title>Neural computation</title><addtitle>Neural Comput</addtitle><description>In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impractical way.</description><subject>Accuracy</subject><subject>Algorithms</subject><subject>Applied sciences</subject><subject>Approximation</subject><subject>Artificial intelligence</subject><subject>Belief networks</subject><subject>Biological and medical sciences</subject><subject>Computer science; control theory; systems</subject><subject>Exact sciences and technology</subject><subject>Fundamental and applied biological sciences. Psychology</subject><subject>General aspects</subject><subject>Humans</subject><subject>Learning</subject><subject>Learning and adaptive systems</subject><subject>Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects)</subject><subject>Miscellaneous</subject><subject>Neural Networks (Computer)</subject><subject>Nonlinear Dynamics</subject><issn>0899-7667</issn><issn>1530-888X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2008</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNqNkUtvFDEQhC1ERJbAL0BCIyQ4ZTbdtsf23FhCHkhROEAkbpbX9iCHmfFg7ybAr49XuwKEeJ368lV1dRchTxDmiIIejd7GOQVQc6Q1yFoIvEdm2DColVIf7pMZqLatpRBynzzM-RoABELzgOyjahhT2M7Iy9feT4fVpUkp3lbvwschBle98n3wXXXpV7cxfcrVIvnqagw3PmXTV4tpSvFLGMwqpvyI7HWmz_7xbh6Qq9OT98fn9cXbszfHi4vaCqZWtQEppUPnBOtM61rfOGQOfMMleGqdWbKOW6takAoAGyuRe6OWS9cJ03aWHZAXW9-y-_Pa55UeQra-783o4zpr0QrkwNk_QYogqOLN_4DFjokCPvsFvI7rNJZrC4MUleK8QGwL2RRzTr7TUyovSl81gt70pTd96U1fGqkGqUtfRfV0Z71eDt790OwKKsDzHWCyNX2XzGhD_s5RkKJBUIU733JD-CneZuUNhYCoGVDeyCKgWOLokuJbmP6Q6eg3Vn-74g6FkMK8</recordid><startdate>20081101</startdate><enddate>20081101</enddate><creator>Sutskever, Ilya</creator><creator>Hinton, Geoffrey E.</creator><general>MIT Press</general><general>MIT Press Journals, The</general><scope>IQODW</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>7TK</scope><scope>7X8</scope></search><sort><creationdate>20081101</creationdate><title>Deep, Narrow Sigmoid Belief Networks Are Universal Approximators</title><author>Sutskever, Ilya ; Hinton, Geoffrey E.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c638t-a0777d1dd63fa9d9e5d13d0e5470e2cdab3f4cc890780015c714ea8bbdf6a9fc3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2008</creationdate><topic>Accuracy</topic><topic>Algorithms</topic><topic>Applied sciences</topic><topic>Approximation</topic><topic>Artificial intelligence</topic><topic>Belief networks</topic><topic>Biological and medical sciences</topic><topic>Computer science; control theory; systems</topic><topic>Exact sciences and technology</topic><topic>Fundamental and applied biological sciences. Psychology</topic><topic>General aspects</topic><topic>Humans</topic><topic>Learning</topic><topic>Learning and adaptive systems</topic><topic>Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects)</topic><topic>Miscellaneous</topic><topic>Neural Networks (Computer)</topic><topic>Nonlinear Dynamics</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Sutskever, Ilya</creatorcontrib><creatorcontrib>Hinton, Geoffrey E.</creatorcontrib><collection>Pascal-Francis</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>Neurosciences Abstracts</collection><collection>MEDLINE - Academic</collection><jtitle>Neural computation</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sutskever, Ilya</au><au>Hinton, Geoffrey E.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Deep, Narrow Sigmoid Belief Networks Are Universal Approximators</atitle><jtitle>Neural computation</jtitle><addtitle>Neural Comput</addtitle><date>2008-11-01</date><risdate>2008</risdate><volume>20</volume><issue>11</issue><spage>2629</spage><epage>2636</epage><pages>2629-2636</pages><issn>0899-7667</issn><eissn>1530-888X</eissn><coden>NEUCEB</coden><abstract>In this note, we show that exponentially deep belief networks can approximate any distribution over binary vectors to arbitrary accuracy, even when the width of each layer is limited to the dimensionality of the data. We further show that such networks can be greedily learned in an easy yet impractical way.</abstract><cop>One Rogers Street, Cambridge, MA 02142-1209, USA</cop><pub>MIT Press</pub><pmid>18533819</pmid><doi>10.1162/neco.2008.12-07-661</doi><tpages>8</tpages><oa>free_for_read</oa></addata></record>
fulltext	fulltext
identifier	ISSN: 0899-7667
ispartof	Neural computation, 2008-11, Vol.20 (11), p.2629-2636
issn	0899-7667 1530-888X
language	eng
recordid	cdi_pubmed_primary_18533819
source	MEDLINE; MIT Press Journals
subjects	Accuracy Algorithms Applied sciences Approximation Artificial intelligence Belief networks Biological and medical sciences Computer science control theory systems Exact sciences and technology Fundamental and applied biological sciences. Psychology General aspects Humans Learning Learning and adaptive systems Mathematics in biology. Statistical analysis. Models. Metrology. Data processing in biology (general aspects) Miscellaneous Neural Networks (Computer) Nonlinear Dynamics
title	Deep, Narrow Sigmoid Belief Networks Are Universal Approximators
url	https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-26T12%3A31%3A17IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Deep,%20Narrow%20Sigmoid%20Belief%20Networks%20Are%20Universal%20Approximators&rft.jtitle=Neural%20computation&rft.au=Sutskever,%20Ilya&rft.date=2008-11-01&rft.volume=20&rft.issue=11&rft.spage=2629&rft.epage=2636&rft.pages=2629-2636&rft.issn=0899-7667&rft.eissn=1530-888X&rft.coden=NEUCEB&rft_id=info:doi/10.1162/neco.2008.12-07-661&rft_dat=%3Cproquest_pubme%3E21043336%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=211218844&rft_id=info:pmid/18533819&rfr_iscdi=true