Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior

We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Cell reports (Cambridge) 2022-08, Vol.40 (9), p.111275-111275, Article 111275
Hauptverfasser: Sakai, Yuki, Sakai, Yutaka, Abe, Yoshinari, Narumoto, Jin, Tanaka, Saori C.
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 111275
container_issue 9
container_start_page 111275
container_title Cell reports (Cambridge)
container_volume 40
creator Sakai, Yuki
Sakai, Yutaka
Abe, Yoshinari
Narumoto, Jin
Tanaka, Saori C.
description We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes. [Display omitted] •We present a computational model of OCD•Imbalanced learning between reinforcement and punishment induces OC behaviors•Patients with OCD show imbalanced learning parameters consistent with our model•Serotonin enhancers normalize imbalanced learning parameters in OCD Sakai et al. show that imbalanced learning between reinforcement and punishment can induce obsessive-compulsive disorder (OCD). A computational model of OCD reveals extremely imbalanced learning parameters in patients with OCD. Moreover, effects of behavioral therapy and psychotropic medication are reflected in the model and behavioral results.
doi_str_mv 10.1016/j.celrep.2022.111275
format Article
fullrecord <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2709017808</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S2211124722010956</els_id><sourcerecordid>2709017808</sourcerecordid><originalsourceid>FETCH-LOGICAL-c359t-d9c8b49fed3fde7521eac0e1cb31b409e6a89724be375c51046ff244995291613</originalsourceid><addsrcrecordid>eNp9kM1OwzAQhCMEElXpG3DwkUuK13GS-oKEKv6kIi5wthxnQ10lcfAmlfoCPDcp4dATe9lZaWal-aLoGvgSOGS3u6XFOmC3FFyIJQCIPD2LZkIAxCBkfn6iL6MF0Y6Pk3EAJWfR9ys2PhxYH4xF5prC1KY9qpYFdG3lg8UG256ZtmTd0Dra_p50oB4bYtacGMd8Vzvrema33lkkVqMpXfvJes98QUjk9hhb33RDfZSswK3ZOx-uoovK1ISLvz2PPh4f3tfP8ebt6WV9v4ltkqo-LpVdFVJVWCZViXkqAI3lCLZIoJBcYWZWKheywCRPbQpcZlUlpFQqFQoySObRzfS3C_5rQOp142jkN5ZGP5AWOVcc8hVfjVY5WW3wRAEr3QXXmHDQwPWRvN7pibw-ktcT-TF2N8VwrLF3GDRZhyPS0gW0vS69-__BD-mtkUM</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2709017808</pqid></control><display><type>article</type><title>Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior</title><source>DOAJ Directory of Open Access Journals</source><source>Cell Press Free Archives</source><source>Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals</source><source>Alma/SFX Local Collection</source><creator>Sakai, Yuki ; Sakai, Yutaka ; Abe, Yoshinari ; Narumoto, Jin ; Tanaka, Saori C.</creator><creatorcontrib>Sakai, Yuki ; Sakai, Yutaka ; Abe, Yoshinari ; Narumoto, Jin ; Tanaka, Saori C.</creatorcontrib><description>We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes. [Display omitted] •We present a computational model of OCD•Imbalanced learning between reinforcement and punishment induces OC behaviors•Patients with OCD show imbalanced learning parameters consistent with our model•Serotonin enhancers normalize imbalanced learning parameters in OCD Sakai et al. show that imbalanced learning between reinforcement and punishment can induce obsessive-compulsive disorder (OCD). A computational model of OCD reveals extremely imbalanced learning parameters in patients with OCD. Moreover, effects of behavioral therapy and psychotropic medication are reflected in the model and behavioral results.</description><identifier>ISSN: 2211-1247</identifier><identifier>EISSN: 2211-1247</identifier><identifier>DOI: 10.1016/j.celrep.2022.111275</identifier><language>eng</language><publisher>Elsevier Inc</publisher><subject>behavioral therapy ; computational model ; credit assignment ; decision making ; eligibility trace ; memory trace ; obsessive-compulsive disorder ; prediction error ; reinforcement learning ; serotonin reuptake inhibitor</subject><ispartof>Cell reports (Cambridge), 2022-08, Vol.40 (9), p.111275-111275, Article 111275</ispartof><rights>2022 The Author(s)</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c359t-d9c8b49fed3fde7521eac0e1cb31b409e6a89724be375c51046ff244995291613</citedby><cites>FETCH-LOGICAL-c359t-d9c8b49fed3fde7521eac0e1cb31b409e6a89724be375c51046ff244995291613</cites></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,776,780,860,27903,27904</link.rule.ids></links><search><creatorcontrib>Sakai, Yuki</creatorcontrib><creatorcontrib>Sakai, Yutaka</creatorcontrib><creatorcontrib>Abe, Yoshinari</creatorcontrib><creatorcontrib>Narumoto, Jin</creatorcontrib><creatorcontrib>Tanaka, Saori C.</creatorcontrib><title>Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior</title><title>Cell reports (Cambridge)</title><description>We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes. [Display omitted] •We present a computational model of OCD•Imbalanced learning between reinforcement and punishment induces OC behaviors•Patients with OCD show imbalanced learning parameters consistent with our model•Serotonin enhancers normalize imbalanced learning parameters in OCD Sakai et al. show that imbalanced learning between reinforcement and punishment can induce obsessive-compulsive disorder (OCD). A computational model of OCD reveals extremely imbalanced learning parameters in patients with OCD. Moreover, effects of behavioral therapy and psychotropic medication are reflected in the model and behavioral results.</description><subject>behavioral therapy</subject><subject>computational model</subject><subject>credit assignment</subject><subject>decision making</subject><subject>eligibility trace</subject><subject>memory trace</subject><subject>obsessive-compulsive disorder</subject><subject>prediction error</subject><subject>reinforcement learning</subject><subject>serotonin reuptake inhibitor</subject><issn>2211-1247</issn><issn>2211-1247</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2022</creationdate><recordtype>article</recordtype><recordid>eNp9kM1OwzAQhCMEElXpG3DwkUuK13GS-oKEKv6kIi5wthxnQ10lcfAmlfoCPDcp4dATe9lZaWal-aLoGvgSOGS3u6XFOmC3FFyIJQCIPD2LZkIAxCBkfn6iL6MF0Y6Pk3EAJWfR9ys2PhxYH4xF5prC1KY9qpYFdG3lg8UG256ZtmTd0Dra_p50oB4bYtacGMd8Vzvrema33lkkVqMpXfvJes98QUjk9hhb33RDfZSswK3ZOx-uoovK1ISLvz2PPh4f3tfP8ebt6WV9v4ltkqo-LpVdFVJVWCZViXkqAI3lCLZIoJBcYWZWKheywCRPbQpcZlUlpFQqFQoySObRzfS3C_5rQOp142jkN5ZGP5AWOVcc8hVfjVY5WW3wRAEr3QXXmHDQwPWRvN7pibw-ktcT-TF2N8VwrLF3GDRZhyPS0gW0vS69-__BD-mtkUM</recordid><startdate>20220830</startdate><enddate>20220830</enddate><creator>Sakai, Yuki</creator><creator>Sakai, Yutaka</creator><creator>Abe, Yoshinari</creator><creator>Narumoto, Jin</creator><creator>Tanaka, Saori C.</creator><general>Elsevier Inc</general><scope>6I.</scope><scope>AAFTH</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope></search><sort><creationdate>20220830</creationdate><title>Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior</title><author>Sakai, Yuki ; Sakai, Yutaka ; Abe, Yoshinari ; Narumoto, Jin ; Tanaka, Saori C.</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c359t-d9c8b49fed3fde7521eac0e1cb31b409e6a89724be375c51046ff244995291613</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2022</creationdate><topic>behavioral therapy</topic><topic>computational model</topic><topic>credit assignment</topic><topic>decision making</topic><topic>eligibility trace</topic><topic>memory trace</topic><topic>obsessive-compulsive disorder</topic><topic>prediction error</topic><topic>reinforcement learning</topic><topic>serotonin reuptake inhibitor</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Sakai, Yuki</creatorcontrib><creatorcontrib>Sakai, Yutaka</creatorcontrib><creatorcontrib>Abe, Yoshinari</creatorcontrib><creatorcontrib>Narumoto, Jin</creatorcontrib><creatorcontrib>Tanaka, Saori C.</creatorcontrib><collection>ScienceDirect Open Access Titles</collection><collection>Elsevier:ScienceDirect:Open Access</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>Cell reports (Cambridge)</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Sakai, Yuki</au><au>Sakai, Yutaka</au><au>Abe, Yoshinari</au><au>Narumoto, Jin</au><au>Tanaka, Saori C.</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior</atitle><jtitle>Cell reports (Cambridge)</jtitle><date>2022-08-30</date><risdate>2022</risdate><volume>40</volume><issue>9</issue><spage>111275</spage><epage>111275</epage><pages>111275-111275</pages><artnum>111275</artnum><issn>2211-1247</issn><eissn>2211-1247</eissn><abstract>We may view most of our daily activities as rational action selections; however, we sometimes reinforce maladaptive behaviors despite having explicit environmental knowledge. In this study, we model obsessive-compulsive disorder (OCD) symptoms as implicitly learned maladaptive behaviors. Simulations in the reinforcement learning framework show that agents implicitly learn to respond to intrusive thoughts when the memory trace signal for past actions decays differently for positive and negative prediction errors. Moreover, this model extends our understanding of therapeutic effects of behavioral therapy in OCD. Using empirical data, we confirm that patients with OCD show extremely imbalanced traces, which are normalized by serotonin enhancers. We find that healthy participants also vary in their obsessive-compulsive tendencies, consistent with the degree of imbalanced traces. These behavioral characteristics can be generalized to variations in the healthy population beyond the spectrum of clinical phenotypes. [Display omitted] •We present a computational model of OCD•Imbalanced learning between reinforcement and punishment induces OC behaviors•Patients with OCD show imbalanced learning parameters consistent with our model•Serotonin enhancers normalize imbalanced learning parameters in OCD Sakai et al. show that imbalanced learning between reinforcement and punishment can induce obsessive-compulsive disorder (OCD). A computational model of OCD reveals extremely imbalanced learning parameters in patients with OCD. Moreover, effects of behavioral therapy and psychotropic medication are reflected in the model and behavioral results.</abstract><pub>Elsevier Inc</pub><doi>10.1016/j.celrep.2022.111275</doi><tpages>1</tpages><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 2211-1247
ispartof Cell reports (Cambridge), 2022-08, Vol.40 (9), p.111275-111275, Article 111275
issn 2211-1247
2211-1247
language eng
recordid cdi_proquest_miscellaneous_2709017808
source DOAJ Directory of Open Access Journals; Cell Press Free Archives; Elektronische Zeitschriftenbibliothek - Frei zugängliche E-Journals; Alma/SFX Local Collection
subjects behavioral therapy
computational model
credit assignment
decision making
eligibility trace
memory trace
obsessive-compulsive disorder
prediction error
reinforcement learning
serotonin reuptake inhibitor
title Memory trace imbalance in reinforcement and punishment systems can reinforce implicit choices leading to obsessive-compulsive behavior
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-26T19%3A58%3A04IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Memory%20trace%20imbalance%20in%20reinforcement%20and%20punishment%20systems%20can%20reinforce%20implicit%20choices%20leading%20to%20obsessive-compulsive%20behavior&rft.jtitle=Cell%20reports%20(Cambridge)&rft.au=Sakai,%20Yuki&rft.date=2022-08-30&rft.volume=40&rft.issue=9&rft.spage=111275&rft.epage=111275&rft.pages=111275-111275&rft.artnum=111275&rft.issn=2211-1247&rft.eissn=2211-1247&rft_id=info:doi/10.1016/j.celrep.2022.111275&rft_dat=%3Cproquest_cross%3E2709017808%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2709017808&rft_id=info:pmid/&rft_els_id=S2211124722010956&rfr_iscdi=true