Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis
OBJECTIVETo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation. METHODSWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 p...
Gespeichert in:
Veröffentlicht in: | Neurology 2019-07, Vol.93 (1), p.e40-e51 |
---|---|
Hauptverfasser: | , , , , , , , , |
Format: | Artikel |
Sprache: | eng |
Schlagworte: | |
Online-Zugang: | Volltext |
Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
container_end_page | e51 |
---|---|
container_issue | 1 |
container_start_page | e40 |
container_title | Neurology |
container_volume | 93 |
creator | Watzlawick, Ralf Antonic, Ana Sena, Emily S Kopp, Marcel A Rind, Julian Dirnagl, Ulrich Macleod, Malcolm Howells, David W Schwab, Jan M |
description | OBJECTIVETo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.
METHODSWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.
RESULTSOverall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3–28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.
CONCLUSIONSWe provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside. |
doi_str_mv | 10.1212/WNL.0000000000007718 |
format | Article |
fullrecord | <record><control><sourceid>proquest_pubme</sourceid><recordid>TN_cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6659001</recordid><sourceformat>XML</sourceformat><sourcesystem>PC</sourcesystem><sourcerecordid>2251105456</sourcerecordid><originalsourceid>FETCH-LOGICAL-c3608-e7109895d7d8bb6ad5ee3b93fa7f971669e73d5ed2ab94438b48e8681b24fa1d3</originalsourceid><addsrcrecordid>eNp9kUtLxDAUhYMoOo7-A5Es3VTzaJPUhSCDLxh0o-gupO2tE22bMWnV-fdGZhR1YTYXcr57cshBaI-SQ8ooO7q_nh6SH0dKqtbQiGZMJIKzh3U0IoSphCupttB2CE-ERFHmm2iLUyozRuQIPdwMfelawDPowbtH6MD2C2y6ChfWBGw7bMqhBwzvc_C2ha43DQ5z28VROl9F4mnwi2N8ilvoTWKisAg27KCN2jQBdldzjO7Oz24nl8n05uJqcjpNSi6ISkBSkqs8q2SlikKYKgPgRc5rI-tcUiFykDxeVswUeZpyVaQKlFC0YGltaMXH6GTpOx-KFqoyBvSm0fOY1fiFdsbq30pnZ_rRvWohsjz-SDQ4WBl49zJA6HVrQwlNYzpwQ9CMZZSSLM1ERNMlWnoXgof6-xlK9GcpOpai_5YS1_Z_Rvxe-mohAmoJvLkmthCem-ENvJ6BafrZ_94f14mazA</addsrcrecordid><sourcetype>Open Access Repository</sourcetype><iscdi>true</iscdi><recordtype>article</recordtype><pqid>2251105456</pqid></control><display><type>article</type><title>Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis</title><source>MEDLINE</source><source>Alma/SFX Local Collection</source><source>Journals@Ovid Complete</source><creator>Watzlawick, Ralf ; Antonic, Ana ; Sena, Emily S ; Kopp, Marcel A ; Rind, Julian ; Dirnagl, Ulrich ; Macleod, Malcolm ; Howells, David W ; Schwab, Jan M</creator><creatorcontrib>Watzlawick, Ralf ; Antonic, Ana ; Sena, Emily S ; Kopp, Marcel A ; Rind, Julian ; Dirnagl, Ulrich ; Macleod, Malcolm ; Howells, David W ; Schwab, Jan M</creatorcontrib><description>OBJECTIVETo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.
METHODSWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.
RESULTSOverall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3–28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.
CONCLUSIONSWe provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.</description><identifier>ISSN: 0028-3878</identifier><identifier>EISSN: 1526-632X</identifier><identifier>DOI: 10.1212/WNL.0000000000007718</identifier><identifier>PMID: 31175207</identifier><language>eng</language><publisher>United States: American Academy of Neurology</publisher><subject>Animals ; Disease Models, Animal ; Publication Bias ; Recovery of Function ; Spinal Cord Injuries - therapy ; Translational Medical Research</subject><ispartof>Neurology, 2019-07, Vol.93 (1), p.e40-e51</ispartof><rights>2019 American Academy of Neurology</rights><rights>2019 American Academy of Neurology.</rights><rights>2019 American Academy of Neurology 2019 American Academy of Neurology</rights><lds50>peer_reviewed</lds50><oa>free_for_read</oa><woscitedreferencessubscribed>false</woscitedreferencessubscribed><cites>FETCH-LOGICAL-c3608-e7109895d7d8bb6ad5ee3b93fa7f971669e73d5ed2ab94438b48e8681b24fa1d3</cites><orcidid>0000-0001-7886-3314 ; 0000-0002-3282-8502 ; 0000-0001-9187-9839</orcidid></display><links><openurl>$$Topenurl_article</openurl><openurlfulltext>$$Topenurlfull_article</openurlfulltext><thumbnail>$$Tsyndetics_thumb_exl</thumbnail><link.rule.ids>230,314,780,784,885,27924,27925</link.rule.ids><backlink>$$Uhttps://www.ncbi.nlm.nih.gov/pubmed/31175207$$D View this record in MEDLINE/PubMed$$Hfree_for_read</backlink></links><search><creatorcontrib>Watzlawick, Ralf</creatorcontrib><creatorcontrib>Antonic, Ana</creatorcontrib><creatorcontrib>Sena, Emily S</creatorcontrib><creatorcontrib>Kopp, Marcel A</creatorcontrib><creatorcontrib>Rind, Julian</creatorcontrib><creatorcontrib>Dirnagl, Ulrich</creatorcontrib><creatorcontrib>Macleod, Malcolm</creatorcontrib><creatorcontrib>Howells, David W</creatorcontrib><creatorcontrib>Schwab, Jan M</creatorcontrib><title>Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis</title><title>Neurology</title><addtitle>Neurology</addtitle><description>OBJECTIVETo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.
METHODSWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.
RESULTSOverall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3–28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.
CONCLUSIONSWe provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.</description><subject>Animals</subject><subject>Disease Models, Animal</subject><subject>Publication Bias</subject><subject>Recovery of Function</subject><subject>Spinal Cord Injuries - therapy</subject><subject>Translational Medical Research</subject><issn>0028-3878</issn><issn>1526-632X</issn><fulltext>true</fulltext><rsrctype>article</rsrctype><creationdate>2019</creationdate><recordtype>article</recordtype><sourceid>EIF</sourceid><recordid>eNp9kUtLxDAUhYMoOo7-A5Es3VTzaJPUhSCDLxh0o-gupO2tE22bMWnV-fdGZhR1YTYXcr57cshBaI-SQ8ooO7q_nh6SH0dKqtbQiGZMJIKzh3U0IoSphCupttB2CE-ERFHmm2iLUyozRuQIPdwMfelawDPowbtH6MD2C2y6ChfWBGw7bMqhBwzvc_C2ha43DQ5z28VROl9F4mnwi2N8ilvoTWKisAg27KCN2jQBdldzjO7Oz24nl8n05uJqcjpNSi6ISkBSkqs8q2SlikKYKgPgRc5rI-tcUiFykDxeVswUeZpyVaQKlFC0YGltaMXH6GTpOx-KFqoyBvSm0fOY1fiFdsbq30pnZ_rRvWohsjz-SDQ4WBl49zJA6HVrQwlNYzpwQ9CMZZSSLM1ERNMlWnoXgof6-xlK9GcpOpai_5YS1_Z_Rvxe-mohAmoJvLkmthCem-ENvJ6BafrZ_94f14mazA</recordid><startdate>20190702</startdate><enddate>20190702</enddate><creator>Watzlawick, Ralf</creator><creator>Antonic, Ana</creator><creator>Sena, Emily S</creator><creator>Kopp, Marcel A</creator><creator>Rind, Julian</creator><creator>Dirnagl, Ulrich</creator><creator>Macleod, Malcolm</creator><creator>Howells, David W</creator><creator>Schwab, Jan M</creator><general>American Academy of Neurology</general><general>Lippincott Williams & Wilkins</general><scope>CGR</scope><scope>CUY</scope><scope>CVF</scope><scope>ECM</scope><scope>EIF</scope><scope>NPM</scope><scope>AAYXX</scope><scope>CITATION</scope><scope>7X8</scope><scope>5PM</scope><orcidid>https://orcid.org/0000-0001-7886-3314</orcidid><orcidid>https://orcid.org/0000-0002-3282-8502</orcidid><orcidid>https://orcid.org/0000-0001-9187-9839</orcidid></search><sort><creationdate>20190702</creationdate><title>Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis</title><author>Watzlawick, Ralf ; Antonic, Ana ; Sena, Emily S ; Kopp, Marcel A ; Rind, Julian ; Dirnagl, Ulrich ; Macleod, Malcolm ; Howells, David W ; Schwab, Jan M</author></sort><facets><frbrtype>5</frbrtype><frbrgroupid>cdi_FETCH-LOGICAL-c3608-e7109895d7d8bb6ad5ee3b93fa7f971669e73d5ed2ab94438b48e8681b24fa1d3</frbrgroupid><rsrctype>articles</rsrctype><prefilter>articles</prefilter><language>eng</language><creationdate>2019</creationdate><topic>Animals</topic><topic>Disease Models, Animal</topic><topic>Publication Bias</topic><topic>Recovery of Function</topic><topic>Spinal Cord Injuries - therapy</topic><topic>Translational Medical Research</topic><toplevel>peer_reviewed</toplevel><toplevel>online_resources</toplevel><creatorcontrib>Watzlawick, Ralf</creatorcontrib><creatorcontrib>Antonic, Ana</creatorcontrib><creatorcontrib>Sena, Emily S</creatorcontrib><creatorcontrib>Kopp, Marcel A</creatorcontrib><creatorcontrib>Rind, Julian</creatorcontrib><creatorcontrib>Dirnagl, Ulrich</creatorcontrib><creatorcontrib>Macleod, Malcolm</creatorcontrib><creatorcontrib>Howells, David W</creatorcontrib><creatorcontrib>Schwab, Jan M</creatorcontrib><collection>Medline</collection><collection>MEDLINE</collection><collection>MEDLINE (Ovid)</collection><collection>MEDLINE</collection><collection>MEDLINE</collection><collection>PubMed</collection><collection>CrossRef</collection><collection>MEDLINE - Academic</collection><collection>PubMed Central (Full Participant titles)</collection><jtitle>Neurology</jtitle></facets><delivery><delcategory>Remote Search Resource</delcategory><fulltext>fulltext</fulltext></delivery><addata><au>Watzlawick, Ralf</au><au>Antonic, Ana</au><au>Sena, Emily S</au><au>Kopp, Marcel A</au><au>Rind, Julian</au><au>Dirnagl, Ulrich</au><au>Macleod, Malcolm</au><au>Howells, David W</au><au>Schwab, Jan M</au><format>journal</format><genre>article</genre><ristype>JOUR</ristype><atitle>Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis</atitle><jtitle>Neurology</jtitle><addtitle>Neurology</addtitle><date>2019-07-02</date><risdate>2019</risdate><volume>93</volume><issue>1</issue><spage>e40</spage><epage>e51</epage><pages>e40-e51</pages><issn>0028-3878</issn><eissn>1526-632X</eissn><abstract>OBJECTIVETo determine whether and to what degree bias and underestimated variability undermine the predictive value of preclinical research for clinical translation.
METHODSWe investigated experimental spinal cord injury (SCI) studies for outcome heterogeneity and the impact of bias. Data from 549 preclinical SCI studies including 9,535 animals were analyzed with meta-regression to assess the effect of various study characteristics and the quality of neurologic recovery.
RESULTSOverall, the included interventions reported a neurobehavioral outcome improvement of 26.3% (95% confidence interval 24.3–28.4). Response to treatment was dependent on experimental modeling paradigms (neurobehavioral score, site of injury, and animal species). Applying multiple outcome measures was consistently associated with smaller effect sizes compared with studies applying only 1 outcome measure. More than half of the studies (51.2%) did not report blinded assessment, constituting a likely source of evaluation bias, with an overstated effect size of 7.2%. Assessment of publication bias, which extrapolates to identify likely missing data, suggested that between 2% and 41% of experiments remain unpublished. Inclusion of these theoretical missing studies suggested an overestimation of efficacy, reducing the effect sizes by between 0.9% and 14.3%.
CONCLUSIONSWe provide empirical evidence of prevalent bias in the design and reporting of experimental SCI studies, resulting in overestimation of the effectiveness. Bias compromises the internal validity and jeopardizes the successful translation of SCI therapies from the bench to bedside.</abstract><cop>United States</cop><pub>American Academy of Neurology</pub><pmid>31175207</pmid><doi>10.1212/WNL.0000000000007718</doi><orcidid>https://orcid.org/0000-0001-7886-3314</orcidid><orcidid>https://orcid.org/0000-0002-3282-8502</orcidid><orcidid>https://orcid.org/0000-0001-9187-9839</orcidid><oa>free_for_read</oa></addata></record> |
fulltext | fulltext |
identifier | ISSN: 0028-3878 |
ispartof | Neurology, 2019-07, Vol.93 (1), p.e40-e51 |
issn | 0028-3878 1526-632X |
language | eng |
recordid | cdi_pubmedcentral_primary_oai_pubmedcentral_nih_gov_6659001 |
source | MEDLINE; Alma/SFX Local Collection; Journals@Ovid Complete |
subjects | Animals Disease Models, Animal Publication Bias Recovery of Function Spinal Cord Injuries - therapy Translational Medical Research |
title | Outcome heterogeneity and bias in acute experimental spinal cord injury: A meta-analysis |
url | https://sfx.bib-bvb.de/sfx_tum?ctx_ver=Z39.88-2004&ctx_enc=info:ofi/enc:UTF-8&ctx_tim=2024-12-25T07%3A39%3A31IST&url_ver=Z39.88-2004&url_ctx_fmt=infofi/fmt:kev:mtx:ctx&rfr_id=info:sid/primo.exlibrisgroup.com:primo3-Article-proquest_pubme&rft_val_fmt=info:ofi/fmt:kev:mtx:journal&rft.genre=article&rft.atitle=Outcome%20heterogeneity%20and%20bias%20in%20acute%20experimental%20spinal%20cord%20injury:%20A%20meta-analysis&rft.jtitle=Neurology&rft.au=Watzlawick,%20Ralf&rft.date=2019-07-02&rft.volume=93&rft.issue=1&rft.spage=e40&rft.epage=e51&rft.pages=e40-e51&rft.issn=0028-3878&rft.eissn=1526-632X&rft_id=info:doi/10.1212/WNL.0000000000007718&rft_dat=%3Cproquest_pubme%3E2251105456%3C/proquest_pubme%3E%3Curl%3E%3C/url%3E&disable_directlink=true&sfx.directlink=off&sfx.report_link=0&rft_id=info:oai/&rft_pqid=2251105456&rft_id=info:pmid/31175207&rfr_iscdi=true |