UNDERMINING THE GOLD STANDARD? THE ROLE OF NON-COMPARATIVE EVIDENCE IN HEALTH TECHNOLOGY ASSESSMENT

OBJECTIVES: Many health technology assessment (HTA) agencies express a preference for 'gold standard' randomized controlled trial (RCT) evidence against a relevant comparator when appraising new health technologies; however, it is not always feasible to design and conduct a clinical trial that meets these criteria. The objective of this study was to assess the current role of non-comparative evidence in HTA decision-making. METHODS: The websites of three HTA agencies: NICE (UK), CADTH (Canada), and IQWiG (Germany) were searched for single HTA reports published between January 2010 and December 2015. The product, indication, outcome, and clinical evidence presented (comparative vs non-comparative) were double-extracted, with any discrepancies reconciled. Multiple technology appraisals, medical devices, vaccination programmes, and requests for advice were excluded. A 'non-comparative' study was denned as any study not presenting results against another treatment (including placebo or best-supportive care), regardless of Phase or setting, including dose-ranging studies. RESULTS: 549 appraisals were extracted. Non-comparative evidence was considered in 38% (45/ 118) NICE submissions, 13% (34/262) CADTH appraisals, and 12% (20/169) IQWiG appraisals. Evidence packages based exclusively on non-comparative evidence were presented in only 4% (5/118) NICE appraisals, 6% CADTH appraisals (16/262), and 4% (6/169) IQWiG appraisals. The majority of drugs appraised solely on the basis of non-comparative evidence were indicated for cancer or hepatitis C. Positive outcome rates (encompassing recommended/restricted/added benefit decisions) for submissions presenting only non-comparative evidence were similar to overall recommendation rates for NICE (84% vs 80%, respectively) and CADTH (68% vs 69%), but were notably (though not statistically) lower for IQWiG (38% vs 22%) (p>0.05 for all). CONCLUSIONS: The role of non-comparative evidence in HTA is currently limited. Non-comparative studies can be viewed as acceptable clinical evidence by NICE and CADTH when these study designs are justifiable and when treatment effect can be convincingly demonstrated using these methods; IQWiG will rarely consider non-comparative studies.