Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials
IMPORTANCE: Many published randomized clinical trials (RCTs) make claims for subgroup differences. OBJECTIVE: To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P
Gespeichert in:
| Veröffentlicht in: | JAMA internal medicine 2017-04, Vol.177 (4), p.554-560 |
|---|---|
| Hauptverfasser: | , , , , , |
| Format: | Artikel |
| Sprache: | eng |
| Schlagworte: | |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| container_end_page | 560 |
|---|---|
| container_issue | 4 |
| container_start_page | 554 |
| container_title | JAMA internal medicine |
| container_volume | 177 |
| creator | Wallach, Joshua D Sullivan, Patrick G Trepanowski, John F Sainani, Kristin L Steyerberg, Ewout W Ioannidis, John P. A |
| description | IMPORTANCE: Many published randomized clinical trials (RCTs) make claims for subgroup differences. OBJECTIVE: To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P |
| doi_str_mv | 10.1001/jamainternmed.2016.9125 |
| format | Article |
| fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6657347</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><ama_id>2601419</ama_id><sourcerecordid>1868393783</sourcerecordid><originalsourceid>FETCH-LOGICAL-a436t-6d2fd5cc8419f4dd10b36df60fd2597465d15ee68cbf91bb589533d763370cbe3</originalsourceid><addsrcrecordid>eNpVkV9PwyAUxYnRuGX6BXzQPvrSCaVQ-mJilvknMTFx85lQoMrSQoV2iX56mdPpeLk33N85kHsAuEBwiiBEVyvRCmN77W2r1TSDiE5LlJEDMM4QZSlFKD_c9ZCOwGkIKxgPgzDH-BiMMobKjFA8Bt18LZpB9MbZxNXJfG2UtlJv-kUfr0NvpGiSxdB1zveJsCqZOe9d5fxOtBiqV--GLpk1wrQhMTZ5jqBrzaeOeGPst8fSG9GEE3BUx6JPf-oEvNzOl7P79PHp7mF285iKHNM-pSqrFZGS5aisc6UQrDBVNYW1ykhZ5JQoRLSmTFZ1iaqKsJJgrAqKcQFlpfEEXG99u6GKa5La9l40vPOmFf6DO2H4_sSaN_7q1pxSUuC8iAaXPwbevQ869Lw1QeqmEVa7IXDEKMMlLhiOaLFFpXcheF3vnkGQbyLje5HxTWR8E1lUnv__5U73G1AEzrZANPibUojiXvAXNOih1w</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>1868393783</pqid></control><display><type>article</type><title>Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials</title><source>MEDLINE</source><source>American Medical Association Journals</source><creator>Wallach, Joshua D ; Sullivan, Patrick G ; Trepanowski, John F ; Sainani, Kristin L ; Steyerberg, Ewout W ; Ioannidis, John P. A</creator><creatorcontrib>Wallach, Joshua D ; Sullivan, Patrick G ; Trepanowski, John F ; Sainani, Kristin L ; Steyerberg, Ewout W ; Ioannidis, John P. A</creatorcontrib><description>IMPORTANCE: Many published randomized clinical trials (RCTs) make claims for subgroup differences. OBJECTIVE: To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses. DATA SOURCES: This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract. STUDY SELECTION: Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings. DATA EXTRACTION AND SYNTHESIS: Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials. MAIN OUTCOMES AND MEASURES: The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses. RESULTS: Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null. CONCLUSIONS AND RELEVANCE: A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup differences are not reproduced.</description><identifier>ISSN: 2168-6106</identifier><identifier>ISSN: 2168-6114</identifier><identifier>EISSN: 2168-6114</identifier><identifier>DOI: 10.1001/jamainternmed.2016.9125</identifier><identifier>PMID: 28192563</identifier><language>eng</language><publisher>United States: American Medical Association</publisher><subject>Data Interpretation, Statistical ; Diagnosis-Related Groups ; Evidence-Based Medicine - methods ; Evidence-Based Medicine - standards ; Humans ; Intention to Treat Analysis ; Publishing - standards ; Randomized Controlled Trials as Topic - methods ; Randomized Controlled Trials as Topic - standards</subject><ispartof>JAMA internal medicine, 2017-04, Vol.177 (4), p.554-560</ispartof><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><citedby>FETCH-LOGICAL-a436t-6d2fd5cc8419f4dd10b36df60fd2597465d15ee68cbf91bb589533d763370cbe3</citedby></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><linktopdf>$$Uhttps://jamanetwork.com/journals/jamainternalmedicine/articlepdf/10.1001/jamainternmed.2016.9125$$EPDF$$P50$$Gama$$H</linktopdf><linktohtml>$$Uhttps://jamanetwork.com/journals/jamainternalmedicine/fullarticle/10.1001/jamainternmed.2016.9125$$EHTML$$P50$$Gama$$H</linktohtml><link.rule.ids>64,230,314,777,781,882,3327,27905,27906,76238,76241</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/28192563$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Wallach, Joshua D</creatorcontrib><creatorcontrib>Sullivan, Patrick G</creatorcontrib><creatorcontrib>Trepanowski, John F</creatorcontrib><creatorcontrib>Sainani, Kristin L</creatorcontrib><creatorcontrib>Steyerberg, Ewout W</creatorcontrib><creatorcontrib>Ioannidis, John P. A</creatorcontrib><title>Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials</title><title>JAMA internal medicine</title><addtitle>JAMA Intern Med</addtitle><description>IMPORTANCE: Many published randomized clinical trials (RCTs) make claims for subgroup differences. OBJECTIVE: To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses. DATA SOURCES: This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract. STUDY SELECTION: Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings. DATA EXTRACTION AND SYNTHESIS: Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials. MAIN OUTCOMES AND MEASURES: The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses. RESULTS: Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null. CONCLUSIONS AND RELEVANCE: A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup differences are not reproduced.</description><subject>Data Interpretation, Statistical</subject><subject>Diagnosis-Related Groups</subject><subject>Evidence-Based Medicine - methods</subject><subject>Evidence-Based Medicine - standards</subject><subject>Humans</subject><subject>Intention to Treat Analysis</subject><subject>Publishing - standards</subject><subject>Randomized Controlled Trials as Topic - methods</subject><subject>Randomized Controlled Trials as Topic - standards</subject><issn>2168-6106</issn><issn>2168-6114</issn><issn>2168-6114</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2017</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNpVkV9PwyAUxYnRuGX6BXzQPvrSCaVQ-mJilvknMTFx85lQoMrSQoV2iX56mdPpeLk33N85kHsAuEBwiiBEVyvRCmN77W2r1TSDiE5LlJEDMM4QZSlFKD_c9ZCOwGkIKxgPgzDH-BiMMobKjFA8Bt18LZpB9MbZxNXJfG2UtlJv-kUfr0NvpGiSxdB1zveJsCqZOe9d5fxOtBiqV--GLpk1wrQhMTZ5jqBrzaeOeGPst8fSG9GEE3BUx6JPf-oEvNzOl7P79PHp7mF285iKHNM-pSqrFZGS5aisc6UQrDBVNYW1ykhZ5JQoRLSmTFZ1iaqKsJJgrAqKcQFlpfEEXG99u6GKa5La9l40vPOmFf6DO2H4_sSaN_7q1pxSUuC8iAaXPwbevQ869Lw1QeqmEVa7IXDEKMMlLhiOaLFFpXcheF3vnkGQbyLje5HxTWR8E1lUnv__5U73G1AEzrZANPibUojiXvAXNOih1w</recordid><startdate>20170401</startdate><enddate>20170401</enddate><creator>Wallach, Joshua D</creator><creator>Sullivan, Patrick G</creator><creator>Trepanowski, John F</creator><creator>Sainani, Kristin L</creator><creator>Steyerberg, Ewout W</creator><creator>Ioannidis, John P. A</creator><general>American Medical Association</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><scope>5PM</scope></search><sort><creationdate>20170401</creationdate><title>Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials</title><author>Wallach, Joshua D ; Sullivan, Patrick G ; Trepanowski, John F ; Sainani, Kristin L ; Steyerberg, Ewout W ; Ioannidis, John P. A</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-a436t-6d2fd5cc8419f4dd10b36df60fd2597465d15ee68cbf91bb589533d763370cbe3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2017</creationdate><topic>Data Interpretation, Statistical</topic><topic>Diagnosis-Related Groups</topic><topic>Evidence-Based Medicine - methods</topic><topic>Evidence-Based Medicine - standards</topic><topic>Humans</topic><topic>Intention to Treat Analysis</topic><topic>Publishing - standards</topic><topic>Randomized Controlled Trials as Topic - methods</topic><topic>Randomized Controlled Trials as Topic - standards</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Wallach, Joshua D</creatorcontrib><creatorcontrib>Sullivan, Patrick G</creatorcontrib><creatorcontrib>Trepanowski, John F</creatorcontrib><creatorcontrib>Sainani, Kristin L</creatorcontrib><creatorcontrib>Steyerberg, Ewout W</creatorcontrib><creatorcontrib>Ioannidis, John P. A</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>JAMA internal medicine</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Wallach, Joshua D</au><au>Sullivan, Patrick G</au><au>Trepanowski, John F</au><au>Sainani, Kristin L</au><au>Steyerberg, Ewout W</au><au>Ioannidis, John P. A</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials</atitle><jtitle>JAMA internal medicine</jtitle><addtitle>JAMA Intern Med</addtitle><date>2017-04-01</date><risdate>2017</risdate><volume>177</volume><issue>4</issue><spage>554</spage><epage>560</epage><pages>554-560</pages><issn>2168-6106</issn><issn>2168-6114</issn><eissn>2168-6114</eissn><abstract>IMPORTANCE: Many published randomized clinical trials (RCTs) make claims for subgroup differences. OBJECTIVE: To evaluate how often subgroup claims reported in the abstracts of RCTs are actually supported by statistical evidence (P < .05 from an interaction test) and corroborated by subsequent RCTs and meta-analyses. DATA SOURCES: This meta-epidemiological survey examines data sets of trials with at least 1 subgroup claim, including Subgroup Analysis of Trials Is Rarely Easy (SATIRE) articles and Discontinuation of Randomized Trials (DISCO) articles. We used Scopus (updated July 2016) to search for English-language articles citing each of the eligible index articles with at least 1 subgroup finding in the abstract. STUDY SELECTION: Articles with a subgroup claim in the abstract with or without evidence of statistical heterogeneity (P < .05 from an interaction test) in the text and articles attempting to corroborate the subgroup findings. DATA EXTRACTION AND SYNTHESIS: Study characteristics of trials with at least 1 subgroup claim in the abstract were recorded. Two reviewers extracted the data necessary to calculate subgroup-level effect sizes, standard errors, and the P values for interaction. For individual RCTs and meta-analyses that attempted to corroborate the subgroup findings from the index articles, trial characteristics were extracted. Cochran Q test was used to reevaluate heterogeneity with the data from all available trials. MAIN OUTCOMES AND MEASURES: The number of subgroup claims in the abstracts of RCTs, the number of subgroup claims in the abstracts of RCTs with statistical support (subgroup findings), and the number of subgroup findings corroborated by subsequent RCTs and meta-analyses. RESULTS: Sixty-four eligible RCTs made a total of 117 subgroup claims in their abstracts. Of these 117 claims, only 46 (39.3%) in 33 articles had evidence of statistically significant heterogeneity from a test for interaction. In addition, out of these 46 subgroup findings, only 16 (34.8%) ensured balance between randomization groups within the subgroups (eg, through stratified randomization), 13 (28.3%) entailed a prespecified subgroup analysis, and 1 (2.2%) was adjusted for multiple testing. Only 5 (10.9%) of the 46 subgroup findings had at least 1 subsequent pure corroboration attempt by a meta-analysis or an RCT. In all 5 cases, the corroboration attempts found no evidence of a statistically significant subgroup effect. In addition, all effect sizes from meta-analyses were attenuated toward the null. CONCLUSIONS AND RELEVANCE: A minority of subgroup claims made in the abstracts of RCTs are supported by their own data (ie, a significant interaction effect). For those that have statistical support (P < .05 from an interaction test), most fail to meet other best practices for subgroup tests, including prespecification, stratified randomization, and adjustment for multiple testing. Attempts to corroborate statistically significant subgroup differences are rare; when done, the initially observed subgroup differences are not reproduced.</abstract><cop>United States</cop><pub>American Medical Association</pub><pmid>28192563</pmid><doi>10.1001/jamainternmed.2016.9125</doi><tpages>7</tpages><oa>free_for_read</oa></addata></record> |
| fulltext | fulltext |
| identifier | ISSN: 2168-6106 |
| ispartof | JAMA internal medicine, 2017-04, Vol.177 (4), p.554-560 |
| issn | 2168-6106 2168-6114 2168-6114 |
| language | eng |
| recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6657347 |
| source | MEDLINE; American Medical Association Journals |
| subjects | Data Interpretation, Statistical Diagnosis-Related Groups Evidence-Based Medicine - methods Evidence-Based Medicine - standards Humans Intention to Treat Analysis Publishing - standards Randomized Controlled Trials as Topic - methods Randomized Controlled Trials as Topic - standards |
| title | Evaluation of Evidence of Statistical Support and Corroboration of Subgroup Claims in Randomized Clinical Trials |
| url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2025-01-20T03%3A41%3A06IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Evaluation%20of%20Evidence%20of%20Statistical%20Support%20and%20Corroboration%20of%20Subgroup%20Claims%20in%20Randomized%20Clinical%20Trials&rft.jtitle=JAMA%20internal%20medicine&rft.au=Wallach,%20Joshua%20D&rft.date=2017-04-01&rft.volume=177&rft.issue=4&rft.spage=554&rft.epage=560&rft.pages=554-560&rft.issn=2168-6106&rft.eissn=2168-6114&rft_id=info:doi/10.1001/jamainternmed.2016.9125&rft_dat=%3Cproquest_pubme%3E1868393783%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=1868393783&rft_id=info:pmid/28192563&rft_ama_id=2601419&rfr_iscdi=true |