We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing
In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review,...
Gespeichert in:
| Veröffentlicht in: | Cortex 2024-03, Vol.172, p.14-37 |
|---|---|
| Hauptverfasser: | , , , , , , , , , , , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 37 |
|---|---|
| container_issue | |
| container_start_page | 14 |
| container_title | Cortex |
| container_volume | 172 |
| creator | Loenneker, Hannah D. Buchanan, Erin M. Martinovici, Ana Primbs, Maximilian A. Elsherif, Mahmoud M. Baker, Bradley J. Dudda, Leonie A. Đurđević, Dušica F. Mišić, Ksenija Peetz, Hannah K. Röer, Jan P. Schulze, Lars Wagner, Lisa Wolska, Julia K. Kührt, Corinna Pronizius, Ekaterina |
| description | In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures. |
| doi_str_mv | 10.1016/j.cortex.2023.11.012 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_cross</sourceid><recordid>TN_cdi_proquest_miscellaneous_2908123943</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><els_id>S0010945223002964</els_id><sourcerecordid>2908123943</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3232-bae4a027ef8674506744084bd598b14fb4d6f5e78dcd990e398eea8fc29d6d933</originalsourceid><addsrcrecordid>eNp9kE1v1DAQhi0EokvhHyDkG1wSxh_Z2BckVPElVeoFxNFy7An1somD7XTpv8fRFo5cxod53nnlh5CXDFoGbP_20LqYCv5uOXDRMtYC44_IjuleNIoBf0x2AAwaLTt-QZ7lfADgoLruKbkQinVS9N2OnL4j9XF-XejPOZ7o6dYWeh9X6oOnR5sLzes0YWrpzUzLLdIwLbXVzg5pHGlJds6LTTgXmnDbhPnHtkhoXQmxZsJUC2yxdEnYLCk6zLlCz8mT0R4zvnh4L8m3jx--Xn1urm8-fbl6f904wQVvBovSAu9xVPtedlCHBCUH32k1MDkO0u_HDnvlndcaUGiFaNXouPZ7r4W4JG_Od2v1rxVzMVPIDo9HO2Ncs-EaFONCyw2VZ9SlmHPC0SwpTDbdGwZmU24O5qzcbMoNY6Yqr7FXDw3rMKH_F_rruALvzgDWf94FTCa7gNWgDwldMT6G_zf8AcXilaE</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2908123943</pqid></control><display><type>article</type><title>We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing</title><source>MEDLINE</source><source>Elsevier ScienceDirect Journals</source><creator>Loenneker, Hannah D. ; Buchanan, Erin M. ; Martinovici, Ana ; Primbs, Maximilian A. ; Elsherif, Mahmoud M. ; Baker, Bradley J. ; Dudda, Leonie A. ; Đurđević, Dušica F. ; Mišić, Ksenija ; Peetz, Hannah K. ; Röer, Jan P. ; Schulze, Lars ; Wagner, Lisa ; Wolska, Julia K. ; Kührt, Corinna ; Pronizius, Ekaterina</creator><creatorcontrib>Loenneker, Hannah D. ; Buchanan, Erin M. ; Martinovici, Ana ; Primbs, Maximilian A. ; Elsherif, Mahmoud M. ; Baker, Bradley J. ; Dudda, Leonie A. ; Đurđević, Dušica F. ; Mišić, Ksenija ; Peetz, Hannah K. ; Röer, Jan P. ; Schulze, Lars ; Wagner, Lisa ; Wolska, Julia K. ; Kührt, Corinna ; Pronizius, Ekaterina</creatorcontrib><description>In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures.</description><identifier>ISSN: 0010-9452</identifier><identifier>ISSN: 1973-8102</identifier><identifier>EISSN: 1973-8102</identifier><identifier>DOI: 10.1016/j.cortex.2023.11.012</identifier><identifier>PMID: 38154375</identifier><language>eng</language><publisher>Italy: Elsevier Ltd</publisher><subject>Checklist ; Expert consensus survey ; Humans ; Multiverse analysis ; Pre-processing ; Reaction time ; Reaction Time - physiology ; Reproducibility of Results ; Research Design - standards ; Transparency</subject><ispartof>Cortex, 2024-03, Vol.172, p.14-37</ispartof><rights>2023</rights><rights>Crown Copyright © 2023. Published by Elsevier Ltd. All rights reserved.</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-c3232-bae4a027ef8674506744084bd598b14fb4d6f5e78dcd990e398eea8fc29d6d933</citedby><cites>FETCH-LOGICAL-c3232-bae4a027ef8674506744084bd598b14fb4d6f5e78dcd990e398eea8fc29d6d933</cites><orcidid>0000-0002-1697-4198 ; 0000-0003-1940-0293 ; 0000-0001-8675-4388 ; 0000-0002-6418-6479 ; 0000-0003-4720-2411 ; 0000-0001-5044-5428 ; 0000-0003-0327-6507 ; 0000-0002-1925-2676 ; 0000-0002-0540-3998 ; 0000-0002-1486-5342 ; 0000-0003-1446-196X ; 0000-0001-7774-3433 ; 0000-0002-3398-5569 ; 0000-0002-9689-4189</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktohtml>$$Uhttps://www.sciencedirect.com/science/article/pii/S0010945223002964$$EHTML$$P50$$Gelsevier$$Hfree_for_read</linktohtml><link.rule.ids>314,776,780,3537,27901,27902,65306</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/38154375$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Loenneker, Hannah D.</creatorcontrib><creatorcontrib>Buchanan, Erin M.</creatorcontrib><creatorcontrib>Martinovici, Ana</creatorcontrib><creatorcontrib>Primbs, Maximilian A.</creatorcontrib><creatorcontrib>Elsherif, Mahmoud M.</creatorcontrib><creatorcontrib>Baker, Bradley J.</creatorcontrib><creatorcontrib>Dudda, Leonie A.</creatorcontrib><creatorcontrib>Đurđević, Dušica F.</creatorcontrib><creatorcontrib>Mišić, Ksenija</creatorcontrib><creatorcontrib>Peetz, Hannah K.</creatorcontrib><creatorcontrib>Röer, Jan P.</creatorcontrib><creatorcontrib>Schulze, Lars</creatorcontrib><creatorcontrib>Wagner, Lisa</creatorcontrib><creatorcontrib>Wolska, Julia K.</creatorcontrib><creatorcontrib>Kührt, Corinna</creatorcontrib><creatorcontrib>Pronizius, Ekaterina</creatorcontrib><title>We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing</title><title>Cortex</title><addtitle>Cortex</addtitle><description>In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures.</description><subject>Checklist</subject><subject>Expert consensus survey</subject><subject>Humans</subject><subject>Multiverse analysis</subject><subject>Pre-processing</subject><subject>Reaction time</subject><subject>Reaction Time - physiology</subject><subject>Reproducibility of Results</subject><subject>Research Design - standards</subject><subject>Transparency</subject><issn>0010-9452</issn><issn>1973-8102</issn><issn>1973-8102</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp9kE1v1DAQhi0EokvhHyDkG1wSxh_Z2BckVPElVeoFxNFy7An1somD7XTpv8fRFo5cxod53nnlh5CXDFoGbP_20LqYCv5uOXDRMtYC44_IjuleNIoBf0x2AAwaLTt-QZ7lfADgoLruKbkQinVS9N2OnL4j9XF-XejPOZ7o6dYWeh9X6oOnR5sLzes0YWrpzUzLLdIwLbXVzg5pHGlJds6LTTgXmnDbhPnHtkhoXQmxZsJUC2yxdEnYLCk6zLlCz8mT0R4zvnh4L8m3jx--Xn1urm8-fbl6f904wQVvBovSAu9xVPtedlCHBCUH32k1MDkO0u_HDnvlndcaUGiFaNXouPZ7r4W4JG_Od2v1rxVzMVPIDo9HO2Ncs-EaFONCyw2VZ9SlmHPC0SwpTDbdGwZmU24O5qzcbMoNY6Yqr7FXDw3rMKH_F_rruALvzgDWf94FTCa7gNWgDwldMT6G_zf8AcXilaE</recordid><startdate>202403</startdate><enddate>202403</enddate><creator>Loenneker, Hannah D.</creator><creator>Buchanan, Erin M.</creator><creator>Martinovici, Ana</creator><creator>Primbs, Maximilian A.</creator><creator>Elsherif, Mahmoud M.</creator><creator>Baker, Bradley J.</creator><creator>Dudda, Leonie A.</creator><creator>Đurđević, Dušica F.</creator><creator>Mišić, Ksenija</creator><creator>Peetz, Hannah K.</creator><creator>Röer, Jan P.</creator><creator>Schulze, Lars</creator><creator>Wagner, Lisa</creator><creator>Wolska, Julia K.</creator><creator>Kührt, Corinna</creator><creator>Pronizius, Ekaterina</creator><general>Elsevier Ltd</general><scope>6I.</scope><scope>AAFTH</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><orcidid>https://orcid.org/0000-0002-1697-4198</orcidid><orcidid>https://orcid.org/0000-0003-1940-0293</orcidid><orcidid>https://orcid.org/0000-0001-8675-4388</orcidid><orcidid>https://orcid.org/0000-0002-6418-6479</orcidid><orcidid>https://orcid.org/0000-0003-4720-2411</orcidid><orcidid>https://orcid.org/0000-0001-5044-5428</orcidid><orcidid>https://orcid.org/0000-0003-0327-6507</orcidid><orcidid>https://orcid.org/0000-0002-1925-2676</orcidid><orcidid>https://orcid.org/0000-0002-0540-3998</orcidid><orcidid>https://orcid.org/0000-0002-1486-5342</orcidid><orcidid>https://orcid.org/0000-0003-1446-196X</orcidid><orcidid>https://orcid.org/0000-0001-7774-3433</orcidid><orcidid>https://orcid.org/0000-0002-3398-5569</orcidid><orcidid>https://orcid.org/0000-0002-9689-4189</orcidid></search><sort><creationdate>202403</creationdate><title>We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing</title><author>Loenneker, Hannah D. ; Buchanan, Erin M. ; Martinovici, Ana ; Primbs, Maximilian A. ; Elsherif, Mahmoud M. ; Baker, Bradley J. ; Dudda, Leonie A. ; Đurđević, Dušica F. ; Mišić, Ksenija ; Peetz, Hannah K. ; Röer, Jan P. ; Schulze, Lars ; Wagner, Lisa ; Wolska, Julia K. ; Kührt, Corinna ; Pronizius, Ekaterina</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3232-bae4a027ef8674506744084bd598b14fb4d6f5e78dcd990e398eea8fc29d6d933</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><topic>Checklist</topic><topic>Expert consensus survey</topic><topic>Humans</topic><topic>Multiverse analysis</topic><topic>Pre-processing</topic><topic>Reaction time</topic><topic>Reaction Time - physiology</topic><topic>Reproducibility of Results</topic><topic>Research Design - standards</topic><topic>Transparency</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Loenneker, Hannah D.</creatorcontrib><creatorcontrib>Buchanan, Erin M.</creatorcontrib><creatorcontrib>Martinovici, Ana</creatorcontrib><creatorcontrib>Primbs, Maximilian A.</creatorcontrib><creatorcontrib>Elsherif, Mahmoud M.</creatorcontrib><creatorcontrib>Baker, Bradley J.</creatorcontrib><creatorcontrib>Dudda, Leonie A.</creatorcontrib><creatorcontrib>Đurđević, Dušica F.</creatorcontrib><creatorcontrib>Mišić, Ksenija</creatorcontrib><creatorcontrib>Peetz, Hannah K.</creatorcontrib><creatorcontrib>Röer, Jan P.</creatorcontrib><creatorcontrib>Schulze, Lars</creatorcontrib><creatorcontrib>Wagner, Lisa</creatorcontrib><creatorcontrib>Wolska, Julia K.</creatorcontrib><creatorcontrib>Kührt, Corinna</creatorcontrib><creatorcontrib>Pronizius, Ekaterina</creatorcontrib><collection>ScienceDirect Open Access Titles</collection><collection>Elsevier:ScienceDirect:Open Access</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><jtitle>Cortex</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Loenneker, Hannah D.</au><au>Buchanan, Erin M.</au><au>Martinovici, Ana</au><au>Primbs, Maximilian A.</au><au>Elsherif, Mahmoud M.</au><au>Baker, Bradley J.</au><au>Dudda, Leonie A.</au><au>Đurđević, Dušica F.</au><au>Mišić, Ksenija</au><au>Peetz, Hannah K.</au><au>Röer, Jan P.</au><au>Schulze, Lars</au><au>Wagner, Lisa</au><au>Wolska, Julia K.</au><au>Kührt, Corinna</au><au>Pronizius, Ekaterina</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing</atitle><jtitle>Cortex</jtitle><addtitle>Cortex</addtitle><date>2024-03</date><risdate>2024</risdate><volume>172</volume><spage>14</spage><epage>37</epage><pages>14-37</pages><issn>0010-9452</issn><issn>1973-8102</issn><eissn>1973-8102</eissn><abstract>In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures.</abstract><cop>Italy</cop><pub>Elsevier Ltd</pub><pmid>38154375</pmid><doi>10.1016/j.cortex.2023.11.012</doi><tpages>24</tpages><orcidid>https://orcid.org/0000-0002-1697-4198</orcidid><orcidid>https://orcid.org/0000-0003-1940-0293</orcidid><orcidid>https://orcid.org/0000-0001-8675-4388</orcidid><orcidid>https://orcid.org/0000-0002-6418-6479</orcidid><orcidid>https://orcid.org/0000-0003-4720-2411</orcidid><orcidid>https://orcid.org/0000-0001-5044-5428</orcidid><orcidid>https://orcid.org/0000-0003-0327-6507</orcidid><orcidid>https://orcid.org/0000-0002-1925-2676</orcidid><orcidid>https://orcid.org/0000-0002-0540-3998</orcidid><orcidid>https://orcid.org/0000-0002-1486-5342</orcidid><orcidid>https://orcid.org/0000-0003-1446-196X</orcidid><orcidid>https://orcid.org/0000-0001-7774-3433</orcidid><orcidid>https://orcid.org/0000-0002-3398-5569</orcidid><orcidid>https://orcid.org/0000-0002-9689-4189</orcidid><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0010-9452 |
| ispartof | Cortex, 2024-03, Vol.172, p.14-37 |
| issn | 0010-9452 1973-8102 1973-8102 |
| language | eng |
| recordid | cdi_proquest_miscellaneous_2908123943 |
| source | MEDLINE; Elsevier ScienceDirect Journals |
| subjects | Checklist Expert consensus survey Humans Multiverse analysis Pre-processing Reaction time Reaction Time - physiology Reproducibility of Results Research Design - standards Transparency |
| title | We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-02-08T22%3A07%3A59IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=We%20don't%20know%20what%20you%20did%20last%20summer.%20On%20the%20importance%20of%20transparent%20reporting%20of%20reaction%20time%20data%20pre-processing&rft.jtitle=Cortex&rft.au=Loenneker,%20Hannah%20D.&rft.date=2024-03&rft.volume=172&rft.spage=14&rft.epage=37&rft.pages=14-37&rft.issn=0010-9452&rft.eissn=1973-8102&rft_id=info:doi/10.1016/j.cortex.2023.11.012&rft_dat=%3Cproquest_cross%3E2908123943%3C/proquest_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2908123943&rft_id=info:pmid/38154375&rft_els_id=S0010945223002964&rfr_iscdi=true |