Perceptions of grant peer reviewers: a mixed methods study
Abstract Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers’ evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after...
Gespeichert in:
| Veröffentlicht in: | Research evaluation 2024-11, Vol.33 |
|---|---|
| Hauptverfasser: | , , , |
| Format: | Artikel |
| Sprache: | eng |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | |
|---|---|
| container_issue | |
| container_start_page | |
| container_title | Research evaluation |
| container_volume | 33 |
| creator | Schmaling, Karen B Evenson, Gabriel R Marble, Blake K Gallo, Stephen A |
| description | Abstract
Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers’ evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations written by the primary reviewer. Comments were coded for topical content and emotional valence blind to participants’ characteristics. The most frequent comments were about their experiences with peer review with negative valence. Additional themes were identified within the content codes, including concerns about favoritism and inappropriate behavior observed in other reviewers. Reviewers who made negative comments gave poorer scores than reviewers who did not. Reviewer mindsets are understudied: negative moods and cognitions may affect reviewers’ overall evaluative severity. Future studies should further investigate these associations. |
| doi_str_mv | 10.1093/reseval/rvae050 |
| format | Article |
| fullrecord | <record><control><sourceid>oup_cross</sourceid><recordid>TN_cdi_crossref_primary_10_1093_reseval_rvae050</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><oup_id>10.1093/reseval/rvae050</oup_id><sourcerecordid>10.1093/reseval/rvae050</sourcerecordid><originalsourceid>FETCH-LOGICAL-c160t-115f0c04c4f48253fb101a0dd7c094ce39a53a68dda736685bf5868d820dfa303</originalsourceid><addsrcrecordid>eNqFj89LwzAYhoMoOKdnrzkLtV-apE12k-EvGOhBzyVLvmhlXcuXrrr_3sl69_Tywvu88DB2LeBWgJU5YcLRbXIaHYKGEzYTqhKZVsqeshlYbbICCnvOLlL6AhCmBDVji1ckj_3QdNvEu8g_yG0H3iMSJxwb_EZKC-542_xg4C0On11IPA27sL9kZ9FtEl5NOWfvD_dvy6ds9fL4vLxbZV6UMGRC6AgelFdRmULLuBYgHIRQebDKo7ROS1eaEFwly9LoddTmUE0BIToJcs7y46-nLiXCWPfUtI72tYD6T72e1OtJ_UDcHIlu1_87_gXHG15b</addsrcrecordid><sourcetype>Aggregation Database</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype></control><display><type>article</type><title>Perceptions of grant peer reviewers: a mixed methods study</title><source>Oxford University Press Journals All Titles (1996-Current)</source><creator>Schmaling, Karen B ; Evenson, Gabriel R ; Marble, Blake K ; Gallo, Stephen A</creator><creatorcontrib>Schmaling, Karen B ; Evenson, Gabriel R ; Marble, Blake K ; Gallo, Stephen A</creatorcontrib><description>Abstract
Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers’ evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations written by the primary reviewer. Comments were coded for topical content and emotional valence blind to participants’ characteristics. The most frequent comments were about their experiences with peer review with negative valence. Additional themes were identified within the content codes, including concerns about favoritism and inappropriate behavior observed in other reviewers. Reviewers who made negative comments gave poorer scores than reviewers who did not. Reviewer mindsets are understudied: negative moods and cognitions may affect reviewers’ overall evaluative severity. Future studies should further investigate these associations.</description><identifier>ISSN: 0958-2029</identifier><identifier>EISSN: 1471-5449</identifier><identifier>DOI: 10.1093/reseval/rvae050</identifier><language>eng</language><publisher>Oxford University Press</publisher><ispartof>Research evaluation, 2024-11, Vol.33</ispartof><rights>The Author(s) 2024. Published by Oxford University Press. All rights reserved. For permissions, please email: journals.permissions@oup.com 2024</rights><lds50>peer_reviewed</lds50><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c160t-115f0c04c4f48253fb101a0dd7c094ce39a53a68dda736685bf5868d820dfa303</cites><orcidid>0000-0001-6043-2153 ; 0000-0003-2085-134X</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>314,780,784,1584,27924,27925</link.rule.ids></links><search><creatorcontrib>Schmaling, Karen B</creatorcontrib><creatorcontrib>Evenson, Gabriel R</creatorcontrib><creatorcontrib>Marble, Blake K</creatorcontrib><creatorcontrib>Gallo, Stephen A</creatorcontrib><title>Perceptions of grant peer reviewers: a mixed methods study</title><title>Research evaluation</title><description>Abstract
Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers’ evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations written by the primary reviewer. Comments were coded for topical content and emotional valence blind to participants’ characteristics. The most frequent comments were about their experiences with peer review with negative valence. Additional themes were identified within the content codes, including concerns about favoritism and inappropriate behavior observed in other reviewers. Reviewers who made negative comments gave poorer scores than reviewers who did not. Reviewer mindsets are understudied: negative moods and cognitions may affect reviewers’ overall evaluative severity. Future studies should further investigate these associations.</description><issn>0958-2029</issn><issn>1471-5449</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2024</creationdate><recordtype>article</recordtype><recordid>eNqFj89LwzAYhoMoOKdnrzkLtV-apE12k-EvGOhBzyVLvmhlXcuXrrr_3sl69_Tywvu88DB2LeBWgJU5YcLRbXIaHYKGEzYTqhKZVsqeshlYbbICCnvOLlL6AhCmBDVji1ckj_3QdNvEu8g_yG0H3iMSJxwb_EZKC-542_xg4C0On11IPA27sL9kZ9FtEl5NOWfvD_dvy6ds9fL4vLxbZV6UMGRC6AgelFdRmULLuBYgHIRQebDKo7ROS1eaEFwly9LoddTmUE0BIToJcs7y46-nLiXCWPfUtI72tYD6T72e1OtJ_UDcHIlu1_87_gXHG15b</recordid><startdate>20241122</startdate><enddate>20241122</enddate><creator>Schmaling, Karen B</creator><creator>Evenson, Gabriel R</creator><creator>Marble, Blake K</creator><creator>Gallo, Stephen A</creator><general>Oxford University Press</general><scope>AAYXX</scope><scope>CITATION</scope><orcidid>https://orcid.org/0000-0001-6043-2153</orcidid><orcidid>https://orcid.org/0000-0003-2085-134X</orcidid></search><sort><creationdate>20241122</creationdate><title>Perceptions of grant peer reviewers: a mixed methods study</title><author>Schmaling, Karen B ; Evenson, Gabriel R ; Marble, Blake K ; Gallo, Stephen A</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c160t-115f0c04c4f48253fb101a0dd7c094ce39a53a68dda736685bf5868d820dfa303</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2024</creationdate><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Schmaling, Karen B</creatorcontrib><creatorcontrib>Evenson, Gabriel R</creatorcontrib><creatorcontrib>Marble, Blake K</creatorcontrib><creatorcontrib>Gallo, Stephen A</creatorcontrib><collection>CrossRef</collection><jtitle>Research evaluation</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Schmaling, Karen B</au><au>Evenson, Gabriel R</au><au>Marble, Blake K</au><au>Gallo, Stephen A</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Perceptions of grant peer reviewers: a mixed methods study</atitle><jtitle>Research evaluation</jtitle><date>2024-11-22</date><risdate>2024</risdate><volume>33</volume><issn>0958-2029</issn><eissn>1471-5449</eissn><abstract>Abstract
Peer review is integral to the evaluation of grant proposals. Reviewer perceptions and characteristics have received limited study, especially their associations with reviewers’ evaluations. This mixed methods study analyzed the unstructured comments of 270 experienced peer reviewers after they scored proposals based on mock overall evaluations written by the primary reviewer. Comments were coded for topical content and emotional valence blind to participants’ characteristics. The most frequent comments were about their experiences with peer review with negative valence. Additional themes were identified within the content codes, including concerns about favoritism and inappropriate behavior observed in other reviewers. Reviewers who made negative comments gave poorer scores than reviewers who did not. Reviewer mindsets are understudied: negative moods and cognitions may affect reviewers’ overall evaluative severity. Future studies should further investigate these associations.</abstract><pub>Oxford University Press</pub><doi>10.1093/reseval/rvae050</doi><orcidid>https://orcid.org/0000-0001-6043-2153</orcidid><orcidid>https://orcid.org/0000-0003-2085-134X</orcidid></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 0958-2029 |
| ispartof | Research evaluation, 2024-11, Vol.33 |
| issn | 0958-2029 1471-5449 |
| language | eng |
| recordid | cdi_crossref_primary_10_1093_reseval_rvae050 |
| source | Oxford University Press Journals All Titles (1996-Current) |
| title | Perceptions of grant peer reviewers: a mixed methods study |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-06T06%3A35%3A19IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-oup_cross&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Perceptions%20of%20grant%20peer%20reviewers:%20a%20mixed%20methods%20study&rft.jtitle=Research%20evaluation&rft.au=Schmaling,%20Karen%20B&rft.date=2024-11-22&rft.volume=33&rft.issn=0958-2029&rft.eissn=1471-5449&rft_id=info:doi/10.1093/reseval/rvae050&rft_dat=%3Coup_cross%3E10.1093/reseval/rvae050%3C/oup_cross%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_id=info:pmid/&rft_oup_id=10.1093/reseval/rvae050&rfr_iscdi=true |