Bellman Error Based Feature Generation using Random Projections on Sparse Spaces

We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, f...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Hauptverfasser:	Fard, Mahdi Milani, Grinberg, Yuri, Farahmand, Amir-massoud, Pineau, Joelle, Precup, Doina
Format:	Artikel
Sprache:	eng
Schlagworte:	Computer Science - Learning Statistics - Machine Learning
Online-Zugang:	Volltext bestellen
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

container_end_page
container_issue
container_start_page
container_title
container_volume
creator	Fard, Mahdi Milani Grinberg, Yuri Farahmand, Amir-massoud Pineau, Joelle Precup, Doina
description	We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections to generate BEBFs for sparse feature spaces. We provide a finite sample analysis of the proposed method, and prove that projections logarithmic in the dimension of the original space are enough to guarantee contraction in the error. Empirical results demonstrate the strength of this method.
doi_str_mv	10.48550/arxiv.1207.5554
format	Article
fullrecord	<record><control><sourceid>arxiv_GOX</sourceid><recordid>TN_cdi_arxiv_primary_1207_5554</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>1207_5554</sourcerecordid><originalsourceid>FETCH-LOGICAL-a654-57258ec5b0e32f051b90a097e297a3e99bddb670ec081825786e39679ca0982a3</originalsourceid><addsrcrecordid>eNotj8FOwzAQRH3hgAp3TpV_IMGxs7F9pFVbkCpRQe_RxtlWQYlTrVsEf08DnJ40MxrpCfFQqLx0AOoR-av7zAutbA4A5a3YLajvB4xyxTyyXGCiVq4JzxcmuaFIjOdujPKSuniUbxjbcZA7Hj8oTHmS1-79hJxoQqB0J24O2Ce6_-dM7Ner_fI5275uXpZP2wwrKDOwGhwFaBQZfVBQNF6h8pa0t2jI-6Ztm8oqCsoVToN1FRlfWR-uK6fRzMT87_bXqD5xNyB_15NZPZmZH29ISEM</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Bellman Error Based Feature Generation using Random Projections on Sparse Spaces</title><source>arXiv.org</source><creator>Fard, Mahdi Milani ; Grinberg, Yuri ; Farahmand, Amir-massoud ; Pineau, Joelle ; Precup, Doina</creator><creatorcontrib>Fard, Mahdi Milani ; Grinberg, Yuri ; Farahmand, Amir-massoud ; Pineau, Joelle ; Precup, Doina</creatorcontrib><description>We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections to generate BEBFs for sparse feature spaces. We provide a finite sample analysis of the proposed method, and prove that projections logarithmic in the dimension of the original space are enough to guarantee contraction in the error. Empirical results demonstrate the strength of this method.</description><identifier>DOI: 10.48550/arxiv.1207.5554</identifier><language>eng</language><subject>Computer Science - Learning ; Statistics - Machine Learning</subject><creationdate>2012-07</creationdate><rights>http://arxiv.org/licenses/nonexclusive-distrib/1.0</rights><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>228,230,780,885</link.rule.ids><linktorsrc>$$Uhttps://arxiv.org/abs/1207.5554$$EView_record_in_Cornell_University$$FView_record_in_$$GCornell_University$$Hfree_for_read</linktorsrc><backlink>$$Uhttps://doi.org/10.48550/arXiv.1207.5554$$DView paper in arXiv$$Hfree_for_read</backlink></links><search><creatorcontrib>Fard, Mahdi Milani</creatorcontrib><creatorcontrib>Grinberg, Yuri</creatorcontrib><creatorcontrib>Farahmand, Amir-massoud</creatorcontrib><creatorcontrib>Pineau, Joelle</creatorcontrib><creatorcontrib>Precup, Doina</creatorcontrib><title>Bellman Error Based Feature Generation using Random Projections on Sparse Spaces</title><description>We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections to generate BEBFs for sparse feature spaces. We provide a finite sample analysis of the proposed method, and prove that projections logarithmic in the dimension of the original space are enough to guarantee contraction in the error. Empirical results demonstrate the strength of this method.</description><subject>Computer Science - Learning</subject><subject>Statistics - Machine Learning</subject><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2012</creationdate><recordtype>article</recordtype><sourceid>GOX</sourceid><recordid>eNotj8FOwzAQRH3hgAp3TpV_IMGxs7F9pFVbkCpRQe_RxtlWQYlTrVsEf08DnJ40MxrpCfFQqLx0AOoR-av7zAutbA4A5a3YLajvB4xyxTyyXGCiVq4JzxcmuaFIjOdujPKSuniUbxjbcZA7Hj8oTHmS1-79hJxoQqB0J24O2Ce6_-dM7Ner_fI5275uXpZP2wwrKDOwGhwFaBQZfVBQNF6h8pa0t2jI-6Ztm8oqCsoVToN1FRlfWR-uK6fRzMT87_bXqD5xNyB_15NZPZmZH29ISEM</recordid><startdate>20120723</startdate><enddate>20120723</enddate><creator>Fard, Mahdi Milani</creator><creator>Grinberg, Yuri</creator><creator>Farahmand, Amir-massoud</creator><creator>Pineau, Joelle</creator><creator>Precup, Doina</creator><scope>AKY</scope><scope>EPD</scope><scope>GOX</scope></search><sort><creationdate>20120723</creationdate><title>Bellman Error Based Feature Generation using Random Projections on Sparse Spaces</title><author>Fard, Mahdi Milani ; Grinberg, Yuri ; Farahmand, Amir-massoud ; Pineau, Joelle ; Precup, Doina</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a654-57258ec5b0e32f051b90a097e297a3e99bddb670ec081825786e39679ca0982a3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2012</creationdate><topic>Computer Science - Learning</topic><topic>Statistics - Machine Learning</topic><toplevel>online_resources</toplevel><creatorcontrib>Fard, Mahdi Milani</creatorcontrib><creatorcontrib>Grinberg, Yuri</creatorcontrib><creatorcontrib>Farahmand, Amir-massoud</creatorcontrib><creatorcontrib>Pineau, Joelle</creatorcontrib><creatorcontrib>Precup, Doina</creatorcontrib><collection>arXiv Computer Science</collection><collection>arXiv Statistics</collection><collection>arXiv.org</collection></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext_linktorsrc</fulltext></delivery><addata><au>Fard, Mahdi Milani</au><au>Grinberg, Yuri</au><au>Farahmand, Amir-massoud</au><au>Pineau, Joelle</au><au>Precup, Doina</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Bellman Error Based Feature Generation using Random Projections on Sparse Spaces</atitle><date>2012-07-23</date><risdate>2012</risdate><abstract>We address the problem of automatic generation of features for value function approximation. Bellman Error Basis Functions (BEBFs) have been shown to improve the error of policy evaluation with function approximation, with a convergence rate similar to that of value iteration. We propose a simple, fast and robust algorithm based on random projections to generate BEBFs for sparse feature spaces. We provide a finite sample analysis of the proposed method, and prove that projections logarithmic in the dimension of the original space are enough to guarantee contraction in the error. Empirical results demonstrate the strength of this method.</abstract><doi>10.48550/arxiv.1207.5554</doi><oa>free_for_read</oa></addata></record>
fulltext	fulltext_linktorsrc
identifier	DOI: 10.48550/arxiv.1207.5554
ispartof
issn
language	eng
recordid	cdi_arxiv_primary_1207_5554
source	arXiv.org
subjects	Computer Science - Learning Statistics - Machine Learning
title	Bellman Error Based Feature Generation using Random Projections on Sparse Spaces
url	https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-12T06%3A29%3A03IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-arxiv_GOX&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Bellman%20Error%20Based%20Feature%20Generation%20using%20Random%20Projections%20on%20Sparse%20Spaces&rft.au=Fard,%20Mahdi%20Milani&rft.date=2012-07-23&rft_id=info:doi/10.48550/arxiv.1207.5554&rft_dat=%3Carxiv_GOX%3E1207_5554%3C/arxiv_GOX%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rfr_iscdi=true