Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances
A new universal estimator of divergence is presented for multidimensional continuous densities based on k -nearest-neighbor ( k -NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In...
Gespeichert in:
| Veröffentlicht in: | IEEE transactions on information theory 2009-05, Vol.55 (5), p.2392-2405 |
|---|---|
| Hauptverfasser: | , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext bestellen |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 2405 |
|---|---|
| container_issue | 5 |
| container_start_page | 2392 |
| container_title | IEEE transactions on information theory |
| container_volume | 55 |
| creator | Qing Wang Kulkarni, S.R. Verdu, S. |
| description | A new universal estimator of divergence is presented for multidimensional continuous densities based on k -nearest-neighbor ( k -NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k -NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k -NN method can be further improved by an adaptive choice of k . |
| doi_str_mv | 10.1109/TIT.2009.2016060 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_RIE</sourceid><recordid>TN_cdi_proquest_miscellaneous_914641652</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ieee_id>4839047</ieee_id><sourcerecordid>914641652</sourcerecordid><originalsourceid>FETCH-LOGICAL-c282t-adc5ab6147472ebb7dd1f72e1ec74ba5d13795406683fc17c85e96262f70e263</originalsourceid><addsrcrecordid>eNpdkM1LAzEQxYMoWKt3wcsiiKetmWw-NkexVQtVL9VryGZna3TbrclW8L83pcWDl8xM8nuPzCPkHOgIgOqb-XQ-YpTqdICkkh6QAQihci0FPyQDSqHMNeflMTmJ8SONXAAbkNex_8awwJXDbBJ7v7S971ZZ04XsadP2vvZLXMV0ZdtsvO16jzF78zb7zJ_RBox9qn7xXiXF2MfeJqd4So4a20Y829chmd9P5neP-ezlYXp3O8sdK1mf29oJW0ngiiuGVaXqGprUATrFKytqKJQWnEpZFo0D5UqBWjLJGkWRyWJIrne269B9bdJXzNJHh21rV9htotHAJQcpWCIv_5Ef3SakpaIBLXQhi0IkiO4gF7oYAzZmHVIg4ccANduQTQrZbEM2-5CT5Grva6OzbRPS-j7-6RgoziTfWl_sOI-If8-8LDTlqvgFYdWEvA</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>195936335</pqid></control><display><type>article</type><title>Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances</title><source>IEEE Electronic Library (IEL)</source><creator>Qing Wang ; Kulkarni, S.R. ; Verdu, S.</creator><creatorcontrib>Qing Wang ; Kulkarni, S.R. ; Verdu, S.</creatorcontrib><description>A new universal estimator of divergence is presented for multidimensional continuous densities based on k -nearest-neighbor ( k -NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k -NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k -NN method can be further improved by an adaptive choice of k .</description><identifier>ISSN: 0018-9448</identifier><identifier>EISSN: 1557-9654</identifier><identifier>DOI: 10.1109/TIT.2009.2016060</identifier><identifier>CODEN: IETTAW</identifier><language>eng</language><publisher>New York, NY: IEEE</publisher><subject>Algorithms ; Applied sciences ; Asymptotic properties ; Convergence ; Density ; Density measurement ; Divergence ; Estimates ; Estimators ; Exact sciences and technology ; Experiments ; Frequency estimation ; information measure ; Information theory ; Information, signal and communications theory ; Kullback-Leibler ; Laboratories ; Methods ; Multidimensional systems ; Mutual information ; nearest-neighbor ; Neuroscience ; partition ; Partitioning algorithms ; Probability distribution ; random vector ; Telecommunications and information theory ; universal estimation</subject><ispartof>IEEE transactions on information theory, 2009-05, Vol.55 (5), p.2392-2405</ispartof><rights>2009 INIST-CNRS</rights><rights>Copyright Institute of Electrical and Electronics Engineers, Inc. (IEEE) May 2009</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c282t-adc5ab6147472ebb7dd1f72e1ec74ba5d13795406683fc17c85e96262f70e263</citedby><cites>FETCH-LOGICAL-c282t-adc5ab6147472ebb7dd1f72e1ec74ba5d13795406683fc17c85e96262f70e263</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://ieeexplore.ieee.org/document/4839047$$EHTML$$P50$$Gieee$$H</linktohtml><link.rule.ids>314,776,780,792,27901,27902,54733</link.rule.ids><linktorsrc>$$Uhttps://ieeexplore.ieee.org/document/4839047$$EView_record_in_IEEE$$FView_record_in_$$GIEEE</linktorsrc><backlink>$$Uhttp://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=21742645$$DView record in Pascal Francis$$Hfree_for_read</backlink></links><search><creatorcontrib>Qing Wang</creatorcontrib><creatorcontrib>Kulkarni, S.R.</creatorcontrib><creatorcontrib>Verdu, S.</creatorcontrib><title>Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances</title><title>IEEE transactions on information theory</title><addtitle>TIT</addtitle><description>A new universal estimator of divergence is presented for multidimensional continuous densities based on k -nearest-neighbor ( k -NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k -NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k -NN method can be further improved by an adaptive choice of k .</description><subject>Algorithms</subject><subject>Applied sciences</subject><subject>Asymptotic properties</subject><subject>Convergence</subject><subject>Density</subject><subject>Density measurement</subject><subject>Divergence</subject><subject>Estimates</subject><subject>Estimators</subject><subject>Exact sciences and technology</subject><subject>Experiments</subject><subject>Frequency estimation</subject><subject>information measure</subject><subject>Information theory</subject><subject>Information, signal and communications theory</subject><subject>Kullback-Leibler</subject><subject>Laboratories</subject><subject>Methods</subject><subject>Multidimensional systems</subject><subject>Mutual information</subject><subject>nearest-neighbor</subject><subject>Neuroscience</subject><subject>partition</subject><subject>Partitioning algorithms</subject><subject>Probability distribution</subject><subject>random vector</subject><subject>Telecommunications and information theory</subject><subject>universal estimation</subject><issn>0018-9448</issn><issn>1557-9654</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2009</creationdate><recordtype>article</recordtype><sourceid>RIE</sourceid><recordid>eNpdkM1LAzEQxYMoWKt3wcsiiKetmWw-NkexVQtVL9VryGZna3TbrclW8L83pcWDl8xM8nuPzCPkHOgIgOqb-XQ-YpTqdICkkh6QAQihci0FPyQDSqHMNeflMTmJ8SONXAAbkNex_8awwJXDbBJ7v7S971ZZ04XsadP2vvZLXMV0ZdtsvO16jzF78zb7zJ_RBox9qn7xXiXF2MfeJqd4So4a20Y829chmd9P5neP-ezlYXp3O8sdK1mf29oJW0ngiiuGVaXqGprUATrFKytqKJQWnEpZFo0D5UqBWjLJGkWRyWJIrne269B9bdJXzNJHh21rV9htotHAJQcpWCIv_5Ef3SakpaIBLXQhi0IkiO4gF7oYAzZmHVIg4ccANduQTQrZbEM2-5CT5Grva6OzbRPS-j7-6RgoziTfWl_sOI-If8-8LDTlqvgFYdWEvA</recordid><startdate>20090501</startdate><enddate>20090501</enddate><creator>Qing Wang</creator><creator>Kulkarni, S.R.</creator><creator>Verdu, S.</creator><general>IEEE</general><general>Institute of Electrical and Electronics Engineers</general><general>The Institute of Electrical and Electronics Engineers, Inc. (IEEE)</general><scope>97E</scope><scope>RIA</scope><scope>RIE</scope><scope>IQODW</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7SC</scope><scope>7SP</scope><scope>8FD</scope><scope>JQ2</scope><scope>L7M</scope><scope>L~C</scope><scope>L~D</scope><scope>F28</scope><scope>FR3</scope></search><sort><creationdate>20090501</creationdate><title>Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances</title><author>Qing Wang ; Kulkarni, S.R. ; Verdu, S.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c282t-adc5ab6147472ebb7dd1f72e1ec74ba5d13795406683fc17c85e96262f70e263</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2009</creationdate><topic>Algorithms</topic><topic>Applied sciences</topic><topic>Asymptotic properties</topic><topic>Convergence</topic><topic>Density</topic><topic>Density measurement</topic><topic>Divergence</topic><topic>Estimates</topic><topic>Estimators</topic><topic>Exact sciences and technology</topic><topic>Experiments</topic><topic>Frequency estimation</topic><topic>information measure</topic><topic>Information theory</topic><topic>Information, signal and communications theory</topic><topic>Kullback-Leibler</topic><topic>Laboratories</topic><topic>Methods</topic><topic>Multidimensional systems</topic><topic>Mutual information</topic><topic>nearest-neighbor</topic><topic>Neuroscience</topic><topic>partition</topic><topic>Partitioning algorithms</topic><topic>Probability distribution</topic><topic>random vector</topic><topic>Telecommunications and information theory</topic><topic>universal estimation</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Qing Wang</creatorcontrib><creatorcontrib>Kulkarni, S.R.</creatorcontrib><creatorcontrib>Verdu, S.</creatorcontrib><collection>IEEE All-Society Periodicals Package (ASPP) 2005-present</collection><collection>IEEE All-Society Periodicals Package (ASPP) 1998-Present</collection><collection>IEEE Electronic Library (IEL)</collection><collection>Pascal-Francis</collection><collection>CrossRef</collection><collection>Computer and Information Systems Abstracts</collection><collection>Electronics & Communications Abstracts</collection><collection>Technology Research Database</collection><collection>ProQuest Computer Science Collection</collection><collection>Advanced Technologies Database with Aerospace</collection><collection>Computer and Information Systems Abstracts Academic</collection><collection>Computer and Information Systems Abstracts Professional</collection><collection>ANTE: Abstracts in New Technology & Engineering</collection><collection>Engineering Research Database</collection><jtitle>IEEE transactions on information theory</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Qing Wang</au><au>Kulkarni, S.R.</au><au>Verdu, S.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances</atitle><jtitle>IEEE transactions on information theory</jtitle><stitle>TIT</stitle><date>2009-05-01</date><risdate>2009</risdate><volume>55</volume><issue>5</issue><spage>2392</spage><epage>2405</epage><pages>2392-2405</pages><issn>0018-9448</issn><eissn>1557-9654</eissn><coden>IETTAW</coden><abstract>A new universal estimator of divergence is presented for multidimensional continuous densities based on k -nearest-neighbor ( k -NN) distances. Assuming independent and identically distributed (i.i.d.) samples, the new estimator is proved to be asymptotically unbiased and mean-square consistent. In experiments with high-dimensional data, the k -NN approach generally exhibits faster convergence than previous algorithms. It is also shown that the speed of convergence of the k -NN method can be further improved by an adaptive choice of k .</abstract><cop>New York, NY</cop><pub>IEEE</pub><doi>10.1109/TIT.2009.2016060</doi><tpages>14</tpages></addata></record> |
| fulltext | fulltext_linktorsrc |
| identifier | ISSN: 0018-9448 |
| ispartof | IEEE transactions on information theory, 2009-05, Vol.55 (5), p.2392-2405 |
| issn | 0018-9448 1557-9654 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_914641652 |
| source | IEEE Electronic Library (IEL) |
| subjects | Algorithms Applied sciences Asymptotic properties Convergence Density Density measurement Divergence Estimates Estimators Exact sciences and technology Experiments Frequency estimation information measure Information theory Information, signal and communications theory Kullback-Leibler Laboratories Methods Multidimensional systems Mutual information nearest-neighbor Neuroscience partition Partitioning algorithms Probability distribution random vector Telecommunications and information theory universal estimation |
| title | Divergence Estimation for Multidimensional Densities Via k-Nearest-Neighbor Distances |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-12T19%3A55%3A22IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_RIE&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Divergence%20Estimation%20for%20Multidimensional%20Densities%20Via%20k-Nearest-Neighbor%20Distances&rft.jtitle=IEEE%20transactions%20on%20information%20theory&rft.au=Qing%20Wang&rft.date=2009-05-01&rft.volume=55&rft.issue=5&rft.spage=2392&rft.epage=2405&rft.pages=2392-2405&rft.issn=0018-9448&rft.eissn=1557-9654&rft.coden=IETTAW&rft_id=info:doi/10.1109/TIT.2009.2016060&rft_dat=%3Cproquest_RIE%3E914641652%3C/proquest_RIE%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=195936335&rft_id=info:pmid/&rft_ieee_id=4839047&rfr_iscdi=true |