Evaluating the accuracy and economic value of a new test in the absence of a perfect reference test

Background Streptococcus pneumoniae (SP) pneumonia is often treated empirically as diagnosis is challenging because of the lack of a perfect test. Using BinaxNOW‐SP, a urinary antigen test, as an add‐on to standard cultures may not only increase diagnostic yield but also increase costs. Objective To estimate the sensitivity and specificity of BinaxNOW‐SP and subsequently estimate the cost‐effectiveness of adding BinaxNOW‐SP to the diagnostic work‐up. Design We fit a Bayesian latent‐class meta‐analysis model to obtain estimates of BinaxNOW‐SP accuracy that adjust for the imperfect accuracy of culture. Meta‐analysis results were combined with information on prevalence of SP pneumonia to estimate the number of patients who are correctly classified under competing diagnostic strategies. Taking into consideration the cost of antibiotics, we determined the incremental cost of adding BinaxNOW‐SP to the work‐up per case correctly diagnosed. Results The BinaxNOW‐SP test had a pooled sensitivity of 0.74 (95% credible interval [CrI], 0.67‐0.83) and a pooled specificity of 0.96 (95% CrI, 0.92‐0.99). An overall increase in diagnostic accuracy of 6.2% due to the addition of BinaxNOW‐SP corresponded to an incremental cost per case correctly classified of $582 Canadian dollars. Conclusions The methods we have described allow us to evaluate the accuracy and economic value of a new test in the absence of a perfect reference test using an evidence‐based approach.