An asymptotic property of model selection criteria
Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation,...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on information theory 1998-01, Vol.44 (1), p.95-116 |
|---|---|
| Hauptverfasser: | , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 116 |
|---|---|
| container_issue | 1 |
| container_start_page | 95 |
| container_title | IEEE transactions on information theory |
| container_volume | 44 |
| creator | Yuhong Yang Barron, A.R. |
| description | Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W/sub 2//sup s/(U) without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided. |
| doi_str_mv | 10.1109/18.650993 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_ieee_primary_650993</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>650993</ieee_id><sourcerecordid>28480016</sourcerecordid><originalsourceid>FETCH-LOGICAL-c370t-c7dd6e6f1aaea9a0bf27b3971c727fd9b20fdf21d68d32a949192ad99fafa553</originalsourceid><addsrcrecordid>eNpdkD1rwzAQhkVpoWnaoWsn06HQwakkW5ZuDKFfEOiSXSjSCRxsy5WUIf--Di4dOh3HPTy89xJyz-iKMQovTK0aQQGqC7JgQsgSGlFfkgWlTJVQ1-qa3KR0mNZaML4gfD0UJp36MYfc2mKMYcSYT0XwRR8cdkXCDm1uw1DY2GaMrbklV950Ce9-55Ls3l53m49y-_X-uVlvS1tJmksrnWuw8cwYNGDo3nO5r0AyK7n0Dvaceuc5c41yFTdQAwNuHIA33ghRLcnTrJ0yfR8xZd23yWLXmQHDMWmuajW90Uzg4z_wEI5xmKJpBgL4pD3bnmfIxpBSRK_H2PYmnjSj-tycZkrPzU3sw8y2iPjH_R5_AMT0aE8</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>195921925</pqid></control><display><type>article</type><title>An asymptotic property of model selection criteria</title><source>IEEE Electronic Library (IEL)</source><creator>Yuhong Yang ; Barron, A.R.</creator><creatorcontrib>Yuhong Yang ; Barron, A.R.</creatorcontrib><description>Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W/sub 2//sup s/(U) without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided.</description><identifier>ISSN: 0018-9448</identifier><identifier>EISSN: 1557-9654</identifier><identifier>DOI: 10.1109/18.650993</identifier><identifier>CODEN: IETTAW</identifier><language>eng</language><publisher>New York: IEEE</publisher><subject>Convergence ; Density functional theory ; Entropy ; Estimating techniques ; Extraterrestrial measurements ; Information processing ; Mathematical models ; Maximum likelihood estimation ; Minimax techniques ; Neural networks ; Parametric statistics ; Polynomials ; Spline</subject><ispartof>IEEE transactions on information theory, 1998-01, Vol.44 (1), p.95-116</ispartof><rights>Copyright Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jan 1998</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c370t-c7dd6e6f1aaea9a0bf27b3971c727fd9b20fdf21d68d32a949192ad99fafa553</citedby><cites>FETCH-LOGICAL-c370t-c7dd6e6f1aaea9a0bf27b3971c727fd9b20fdf21d68d32a949192ad99fafa553</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/650993$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,780,784,796,27924,27925,54758</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/650993$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc></links><search><creatorcontrib>Yuhong Yang</creatorcontrib><creatorcontrib>Barron, A.R.</creatorcontrib><title>An asymptotic property of model selection criteria</title><title>IEEE transactions on information theory</title><addtitle>TIT</addtitle><description>Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W/sub 2//sup s/(U) without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided.</description><subject>Convergence</subject><subject>Density functional theory</subject><subject>Entropy</subject><subject>Estimating techniques</subject><subject>Extraterrestrial measurements</subject><subject>Information processing</subject><subject>Mathematical models</subject><subject>Maximum likelihood estimation</subject><subject>Minimax techniques</subject><subject>Neural networks</subject><subject>Parametric statistics</subject><subject>Polynomials</subject><subject>Spline</subject><issn>0018-9448</issn><issn>1557-9654</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>1998</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpdkD1rwzAQhkVpoWnaoWsn06HQwakkW5ZuDKFfEOiSXSjSCRxsy5WUIf--Di4dOh3HPTy89xJyz-iKMQovTK0aQQGqC7JgQsgSGlFfkgWlTJVQ1-qa3KR0mNZaML4gfD0UJp36MYfc2mKMYcSYT0XwRR8cdkXCDm1uw1DY2GaMrbklV950Ce9-55Ls3l53m49y-_X-uVlvS1tJmksrnWuw8cwYNGDo3nO5r0AyK7n0Dvaceuc5c41yFTdQAwNuHIA33ghRLcnTrJ0yfR8xZd23yWLXmQHDMWmuajW90Uzg4z_wEI5xmKJpBgL4pD3bnmfIxpBSRK_H2PYmnjSj-tycZkrPzU3sw8y2iPjH_R5_AMT0aE8</recordid><startdate>199801</startdate><enddate>199801</enddate><creator>Yuhong Yang</creator><creator>Barron, A.R.</creator><general>IEEE</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>RIA</scope><scope>RIE</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope></search><sort><creationdate>199801</creationdate><title>An asymptotic property of model selection criteria</title><author>Yuhong Yang ; Barron, A.R.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c370t-c7dd6e6f1aaea9a0bf27b3971c727fd9b20fdf21d68d32a949192ad99fafa553</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>1998</creationdate><topic>Convergence</topic><topic>Density functional theory</topic><topic>Entropy</topic><topic>Estimating techniques</topic><topic>Extraterrestrial measurements</topic><topic>Information processing</topic><topic>Mathematical models</topic><topic>Maximum likelihood estimation</topic><topic>Minimax techniques</topic><topic>Neural networks</topic><topic>Parametric statistics</topic><topic>Polynomials</topic><topic>Spline</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Yuhong Yang</creatorcontrib><creatorcontrib>Barron, A.R.</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><jtitle>IEEE transactions on information theory</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Yuhong Yang</au><au>Barron, A.R.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>An asymptotic property of model selection criteria</atitle><jtitle>IEEE transactions on information theory</jtitle><stitle>TIT</stitle><date>1998-01</date><risdate>1998</risdate><volume>44</volume><issue>1</issue><spage>95</spage><epage>116</epage><pages>95-116</pages><issn>0018-9448</issn><eissn>1557-9654</eissn><coden>IETTAW</coden><abstract>Probability models are estimated by use of penalized log-likelihood criteria related to Akaike (1973) information criterion (AIC) and minimum description length (MDL). The accuracies of the density estimators are shown to be related to the tradeoff between three terms: the accuracy of approximation, the model dimension, and the descriptive complexity of the model classes. The asymptotic risk is determined under conditions on the penalty term, and is shown to be minimax optimal for some cases. As an application, we show that the optimal rate of convergence is simultaneously achieved for log-densities in Sobolev spaces W/sub 2//sup s/(U) without knowing the smoothness parameter s and norm parameter U in advance. Applications to neural network models and sparse density function estimation are also provided.</abstract><cop>New York</cop><pub>IEEE</pub><doi>10.1109/18.650993</doi><tpages>22</tpages></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | ISSN: 0018-9448 |
| ispartof | IEEE transactions on information theory, 1998-01, Vol.44 (1), p.95-116 |
| issn | 0018-9448 1557-9654 |
| language | eng |
| recordid | cdi_ieee_primary_650993 |
| source | IEEE Electronic Library (IEL) |
| subjects | Convergence Density functional theory Entropy Estimating techniques Extraterrestrial measurements Information processing Mathematical models Maximum likelihood estimation Minimax techniques Neural networks Parametric statistics Polynomials Spline |
| title | An asymptotic property of model selection criteria |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-07T10%3A44%3A47IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=An%20asymptotic%20property%20of%20model%20selection%20criteria&rft.jtitle=IEEE%20transactions%20on%20information%20theory&rft.au=Yuhong%20Yang&rft.date=1998-01&rft.volume=44&rft.issue=1&rft.spage=95&rft.epage=116&rft.pages=95-116&rft.issn=0018-9448&rft.eissn=1557-9654&rft.coden=IETTAW&rft_id=info:doi/10.1109/18.650993&rft_dat=%3Cproquest_RIE%3E28480016%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=195921925&rft_id=info:pmid/&rft_ieee_id=650993&rfr_iscdi=true |