Effects of training on quality of peer review: randomised controlled trial

Abstract Objective To determine the effects of training on the quality of peer review. Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group. Setting and participants Reviewers at a general medical journal. Interventio...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:BMJ 2004-03, Vol.328 (7441), p.673-675
Hauptverfasser: Schroter, Sara, Black, Nick, Evans, Stephen, Carpenter, James, Godlee, Fiona, Smith, Richard
Format: Artikel
Sprache:eng
Schlagworte:
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
container_end_page 675
container_issue 7441
container_start_page 673
container_title BMJ
container_volume 328
creator Schroter, Sara
Black, Nick
Evans, Stephen
Carpenter, James
Godlee, Fiona
Smith, Richard
description Abstract Objective To determine the effects of training on the quality of peer review. Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group. Setting and participants Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials. Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts. Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P < 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002). Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.
doi_str_mv 10.1136/bmj.38023.700775.AE
format Article
fullrecord <record><control><sourceid>jstor_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_381220</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><jstor_id>41707167</jstor_id><sourcerecordid>41707167</sourcerecordid><originalsourceid>FETCH-LOGICAL-b665t-18101544e21c73ecba2bf1a2a253c21fb1147638b1b606475757657f8c2303643</originalsourceid><addsrcrecordid>eNqNkV1v0zAUhi3ExKptvwCBIhDcJfjY8UeQuKiqjg9VII2vS8txneGSxp2dDPbvcUjVARIC-cKW3-fY7zkvQvcBFwCUP6u3m4JKTGghMBaCFfPlHTSDksucSUrvohmuWJVLoPIYncW4wTjBQlac3UPHUFYV55WcoTfLprGmj5lvsj5o17nuMvNddjXo1vU34_XO2pAFe-3st-dZ0N3ab12068z4rg--bdOxD063p-io0W20Z_v9BH08X35YvMpX716-XsxXec0563OQgIGVpSVgBLWm1qRuQBNNGDUEmhqgFJzKGmqOeSlYWpyJRhpCMeUlPUEvpnd3Q721a2OTDd2qXXBbHW6U1079rnTui7r014pKIASn-qf7-uCvBht7lfoxtm11Z_0QlQAhCDDxT5AJkJQxlsBHf4AbP4QuDUERXGLAVTXafvw3KH0oOKQueaLoRJngYwy2OfQFWI3Jq5S8-pm8mpJX82WqevjrSG5r9jkn4Mke0NHotkkxGhdvOcaEwGLkHkzcJvY-HPQSkgp8HEk-6S729vtB1-GrSmoy8_bTQp3zi4vP71dcLRJfTPzo-n86-QF53twB</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1777617656</pqid></control><display><type>article</type><title>Effects of training on quality of peer review: randomised controlled trial</title><source>MEDLINE</source><source>Applied Social Sciences Index &amp; Abstracts (ASSIA)</source><source>JSTOR Archive Collection A-Z Listing</source><source>Alma/SFX Local Collection</source><creator>Schroter, Sara ; Black, Nick ; Evans, Stephen ; Carpenter, James ; Godlee, Fiona ; Smith, Richard</creator><creatorcontrib>Schroter, Sara ; Black, Nick ; Evans, Stephen ; Carpenter, James ; Godlee, Fiona ; Smith, Richard</creatorcontrib><description>Abstract Objective To determine the effects of training on the quality of peer review. Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group. Setting and participants Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials. Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts. Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P &lt; 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002). Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.</description><edition>International edition</edition><identifier>ISSN: 0959-8138</identifier><identifier>ISSN: 0959-8146</identifier><identifier>ISSN: 0959-535X</identifier><identifier>EISSN: 1468-5833</identifier><identifier>EISSN: 1756-1833</identifier><identifier>DOI: 10.1136/bmj.38023.700775.AE</identifier><identifier>PMID: 14996698</identifier><identifier>CODEN: BMJOAE</identifier><language>eng</language><publisher>London: British Medical Journal Publishing Group</publisher><subject>Biological and medical sciences ; Confidence intervals ; Continuing education ; Control groups ; Covariance ; Education, Professional ; Epidemiology ; Experimentation ; General aspects ; Health care outcome assessment ; Humans ; Intervention ; Medical research ; Medical sciences ; Methods ; Observer Variation ; Peer review ; Peer Review, Research - standards ; Peer reviews ; Physicians ; Professional Competence - standards ; Quality ; Quality Control ; Range errors ; Recommendations ; Single blind randomized controlled trials ; Single-Blind Method ; Studies ; Training ; Trajectory control ; Transponders</subject><ispartof>BMJ, 2004-03, Vol.328 (7441), p.673-675</ispartof><rights>2004 BMJ Publishing Group Ltd.</rights><rights>BMJ Publishing Group Ltd 2004</rights><rights>2004 INIST-CNRS</rights><rights>Copyright: 2004 (c) 2004 BMJ Publishing Group Ltd.</rights><rights>Copyright BMJ Publishing Group Mar 20, 2004</rights><rights>Copyright © 2004, BMJ Publishing Group Ltd. 2004</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-b665t-18101544e21c73ecba2bf1a2a253c21fb1147638b1b606475757657f8c2303643</citedby></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://www.jstor.org/stable/pdf/41707167$$EPDF$$P50$$Gjstor$$H</linktopdf><linktohtml>$$Uhttps://www.jstor.org/stable/41707167$$EHTML$$P50$$Gjstor$$H</linktohtml><link.rule.ids>230,314,780,784,803,885,27924,27925,30999,31000,58017,58250</link.rule.ids><backlink>$$Uhttp://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&amp;idt=15577078$$DView record in Pascal Francis$$Hfree_for_read</backlink><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/14996698$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Schroter, Sara</creatorcontrib><creatorcontrib>Black, Nick</creatorcontrib><creatorcontrib>Evans, Stephen</creatorcontrib><creatorcontrib>Carpenter, James</creatorcontrib><creatorcontrib>Godlee, Fiona</creatorcontrib><creatorcontrib>Smith, Richard</creatorcontrib><title>Effects of training on quality of peer review: randomised controlled trial</title><title>BMJ</title><addtitle>BMJ</addtitle><description>Abstract Objective To determine the effects of training on the quality of peer review. Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group. Setting and participants Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials. Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts. Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P &lt; 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002). Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.</description><subject>Biological and medical sciences</subject><subject>Confidence intervals</subject><subject>Continuing education</subject><subject>Control groups</subject><subject>Covariance</subject><subject>Education, Professional</subject><subject>Epidemiology</subject><subject>Experimentation</subject><subject>General aspects</subject><subject>Health care outcome assessment</subject><subject>Humans</subject><subject>Intervention</subject><subject>Medical research</subject><subject>Medical sciences</subject><subject>Methods</subject><subject>Observer Variation</subject><subject>Peer review</subject><subject>Peer Review, Research - standards</subject><subject>Peer reviews</subject><subject>Physicians</subject><subject>Professional Competence - standards</subject><subject>Quality</subject><subject>Quality Control</subject><subject>Range errors</subject><subject>Recommendations</subject><subject>Single blind randomized controlled trials</subject><subject>Single-Blind Method</subject><subject>Studies</subject><subject>Training</subject><subject>Trajectory control</subject><subject>Transponders</subject><issn>0959-8138</issn><issn>0959-8146</issn><issn>0959-535X</issn><issn>1468-5833</issn><issn>1756-1833</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2004</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><sourceid>8G5</sourceid><sourceid>ABUWG</sourceid><sourceid>AFKRA</sourceid><sourceid>AZQEC</sourceid><sourceid>BENPR</sourceid><sourceid>CCPQU</sourceid><sourceid>DWQXO</sourceid><sourceid>GNUQQ</sourceid><sourceid>GUQSH</sourceid><sourceid>M2O</sourceid><sourceid>7QJ</sourceid><recordid>eNqNkV1v0zAUhi3ExKptvwCBIhDcJfjY8UeQuKiqjg9VII2vS8txneGSxp2dDPbvcUjVARIC-cKW3-fY7zkvQvcBFwCUP6u3m4JKTGghMBaCFfPlHTSDksucSUrvohmuWJVLoPIYncW4wTjBQlac3UPHUFYV55WcoTfLprGmj5lvsj5o17nuMvNddjXo1vU34_XO2pAFe-3st-dZ0N3ab12068z4rg--bdOxD063p-io0W20Z_v9BH08X35YvMpX716-XsxXec0563OQgIGVpSVgBLWm1qRuQBNNGDUEmhqgFJzKGmqOeSlYWpyJRhpCMeUlPUEvpnd3Q721a2OTDd2qXXBbHW6U1079rnTui7r014pKIASn-qf7-uCvBht7lfoxtm11Z_0QlQAhCDDxT5AJkJQxlsBHf4AbP4QuDUERXGLAVTXafvw3KH0oOKQueaLoRJngYwy2OfQFWI3Jq5S8-pm8mpJX82WqevjrSG5r9jkn4Mke0NHotkkxGhdvOcaEwGLkHkzcJvY-HPQSkgp8HEk-6S729vtB1-GrSmoy8_bTQp3zi4vP71dcLRJfTPzo-n86-QF53twB</recordid><startdate>20040320</startdate><enddate>20040320</enddate><creator>Schroter, Sara</creator><creator>Black, Nick</creator><creator>Evans, Stephen</creator><creator>Carpenter, James</creator><creator>Godlee, Fiona</creator><creator>Smith, Richard</creator><general>British Medical Journal Publishing Group</general><general>BMJ Publishing Group</general><general>British Medical Association</general><general>BMJ Publishing Group LTD</general><general>BMJ Publishing Group Ltd</general><scope>BSCLL</scope><scope>IQODW</scope><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>3V.</scope><scope>7X7</scope><scope>7XB</scope><scope>88I</scope><scope>8AF</scope><scope>8FE</scope><scope>8FH</scope><scope>8FI</scope><scope>8FJ</scope><scope>8FK</scope><scope>8G5</scope><scope>ABUWG</scope><scope>AFKRA</scope><scope>ASE</scope><scope>AZQEC</scope><scope>BBNVY</scope><scope>BENPR</scope><scope>BHPHI</scope><scope>BTHHO</scope><scope>CCPQU</scope><scope>DWQXO</scope><scope>FPQ</scope><scope>FYUFA</scope><scope>GHDGH</scope><scope>GNUQQ</scope><scope>GUQSH</scope><scope>HCIFZ</scope><scope>K6X</scope><scope>K9.</scope><scope>LK8</scope><scope>M2O</scope><scope>M2P</scope><scope>M7P</scope><scope>MBDVC</scope><scope>PQEST</scope><scope>PQQKQ</scope><scope>PQUKI</scope><scope>PRINS</scope><scope>Q9U</scope><scope>7QJ</scope><scope>7X8</scope><scope>5PM</scope></search><sort><creationdate>20040320</creationdate><title>Effects of training on quality of peer review: randomised controlled trial</title><author>Schroter, Sara ; Black, Nick ; Evans, Stephen ; Carpenter, James ; Godlee, Fiona ; Smith, Richard</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-b665t-18101544e21c73ecba2bf1a2a253c21fb1147638b1b606475757657f8c2303643</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2004</creationdate><topic>Biological and medical sciences</topic><topic>Confidence intervals</topic><topic>Continuing education</topic><topic>Control groups</topic><topic>Covariance</topic><topic>Education, Professional</topic><topic>Epidemiology</topic><topic>Experimentation</topic><topic>General aspects</topic><topic>Health care outcome assessment</topic><topic>Humans</topic><topic>Intervention</topic><topic>Medical research</topic><topic>Medical sciences</topic><topic>Methods</topic><topic>Observer Variation</topic><topic>Peer review</topic><topic>Peer Review, Research - standards</topic><topic>Peer reviews</topic><topic>Physicians</topic><topic>Professional Competence - standards</topic><topic>Quality</topic><topic>Quality Control</topic><topic>Range errors</topic><topic>Recommendations</topic><topic>Single blind randomized controlled trials</topic><topic>Single-Blind Method</topic><topic>Studies</topic><topic>Training</topic><topic>Trajectory control</topic><topic>Transponders</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Schroter, Sara</creatorcontrib><creatorcontrib>Black, Nick</creatorcontrib><creatorcontrib>Evans, Stephen</creatorcontrib><creatorcontrib>Carpenter, James</creatorcontrib><creatorcontrib>Godlee, Fiona</creatorcontrib><creatorcontrib>Smith, Richard</creatorcontrib><collection>Istex</collection><collection>Pascal-Francis</collection><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>ProQuest Central (Corporate)</collection><collection>Health &amp; Medical Collection</collection><collection>ProQuest Central (purchase pre-March 2016)</collection><collection>Science Database (Alumni Edition)</collection><collection>STEM Database</collection><collection>ProQuest SciTech Collection</collection><collection>ProQuest Natural Science Collection</collection><collection>Hospital Premium Collection</collection><collection>Hospital Premium Collection (Alumni Edition)</collection><collection>ProQuest Central (Alumni) (purchase pre-March 2016)</collection><collection>Research Library (Alumni Edition)</collection><collection>ProQuest Central (Alumni Edition)</collection><collection>ProQuest Central UK/Ireland</collection><collection>British Nursing Index</collection><collection>ProQuest Central Essentials</collection><collection>Biological Science Collection</collection><collection>ProQuest Central</collection><collection>Natural Science Collection</collection><collection>BMJ Journals</collection><collection>ProQuest One Community College</collection><collection>ProQuest Central Korea</collection><collection>British Nursing Index (BNI) (1985 to Present)</collection><collection>Health Research Premium Collection</collection><collection>Health Research Premium Collection (Alumni)</collection><collection>ProQuest Central Student</collection><collection>Research Library Prep</collection><collection>SciTech Premium Collection</collection><collection>British Nursing Index</collection><collection>ProQuest Health &amp; Medical Complete (Alumni)</collection><collection>ProQuest Biological Science Collection</collection><collection>Research Library</collection><collection>Science Database</collection><collection>Biological Science Database</collection><collection>Research Library (Corporate)</collection><collection>ProQuest One Academic Eastern Edition (DO NOT USE)</collection><collection>ProQuest One Academic</collection><collection>ProQuest One Academic UKI Edition</collection><collection>ProQuest Central China</collection><collection>ProQuest Central Basic</collection><collection>Applied Social Sciences Index &amp; Abstracts (ASSIA)</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>BMJ</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Schroter, Sara</au><au>Black, Nick</au><au>Evans, Stephen</au><au>Carpenter, James</au><au>Godlee, Fiona</au><au>Smith, Richard</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Effects of training on quality of peer review: randomised controlled trial</atitle><jtitle>BMJ</jtitle><addtitle>BMJ</addtitle><date>2004-03-20</date><risdate>2004</risdate><volume>328</volume><issue>7441</issue><spage>673</spage><epage>675</epage><pages>673-675</pages><issn>0959-8138</issn><issn>0959-8146</issn><issn>0959-535X</issn><eissn>1468-5833</eissn><eissn>1756-1833</eissn><coden>BMJOAE</coden><abstract>Abstract Objective To determine the effects of training on the quality of peer review. Design Single blind randomised controlled trial with two intervention groups receiving different types of training plus a control group. Setting and participants Reviewers at a general medical journal. Interventions Attendance at a training workshop or reception of a self taught training package focusing on what editors want from reviewers and how to critically appraise randomised controlled trials. Main outcome measures Quality of reviews of three manuscripts sent to reviewers at four to six monthly intervals, evaluated using the validated review quality instrument; number of deliberate major errors identified; time taken to review the manuscripts; proportion recommending rejection of the manuscripts. Results Reviewers in the self taught group scored higher in review quality after training than did the control group (score 2.85 v 2.56; difference 0.29, 95% confidence interval 0.14 to 0.44; P = 0.001), but the difference was not of editorial significance and was not maintained in the long term. Both intervention groups identified significantly more major errors after training than did the control group (3.14 and 2.96 v 2.13; P &lt; 0.001), and this remained significant after the reviewers' performance at baseline assessment was taken into account. The evidence for benefit of training was no longer apparent on further testing six months after the interventions. Training had no impact on the time taken to review the papers but was associated with an increased likelihood of recommending rejection (92% and 84% v 76%; P = 0.002). Conclusions Short training packages have only a slight impact on the quality of peer review. The value of longer interventions needs to be assessed.</abstract><cop>London</cop><pub>British Medical Journal Publishing Group</pub><pmid>14996698</pmid><doi>10.1136/bmj.38023.700775.AE</doi><tpages>3</tpages><edition>International edition</edition><oa>free_for_read</oa></addata></record>
fulltext fulltext
identifier ISSN: 0959-8138
ispartof BMJ, 2004-03, Vol.328 (7441), p.673-675
issn 0959-8138
0959-8146
0959-535X
1468-5833
1756-1833
language eng
recordid cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_381220
source MEDLINE; Applied Social Sciences Index & Abstracts (ASSIA); JSTOR Archive Collection A-Z Listing; Alma/SFX Local Collection
subjects Biological and medical sciences
Confidence intervals
Continuing education
Control groups
Covariance
Education, Professional
Epidemiology
Experimentation
General aspects
Health care outcome assessment
Humans
Intervention
Medical research
Medical sciences
Methods
Observer Variation
Peer review
Peer Review, Research - standards
Peer reviews
Physicians
Professional Competence - standards
Quality
Quality Control
Range errors
Recommendations
Single blind randomized controlled trials
Single-Blind Method
Studies
Training
Trajectory control
Transponders
title Effects of training on quality of peer review: randomised controlled trial
url https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-29T13%3A32%3A22IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-jstor_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Effects%20of%20training%20on%20quality%20of%20peer%20review:%20randomised%20controlled%20trial&rft.jtitle=BMJ&rft.au=Schroter,%20Sara&rft.date=2004-03-20&rft.volume=328&rft.issue=7441&rft.spage=673&rft.epage=675&rft.pages=673-675&rft.issn=0959-8138&rft.eissn=1468-5833&rft.coden=BMJOAE&rft_id=info:doi/10.1136/bmj.38023.700775.AE&rft_dat=%3Cjstor_pubme%3E41707167%3C/jstor_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1777617656&rft_id=info:pmid/14996698&rft_jstor_id=41707167&rfr_iscdi=true