Accuracy of Administrative Database Algorithms for Hospitalized Pneumonia in Adults: a Systematic Review

Background Administrative data algorithms (ADAs) to identify pneumonia cases are commonly used in the analysis of pneumonia burden, trends, etiology, processes of care, outcomes, health care utilization, cost, and response to preventative and therapeutic interventions. However, without a good understanding of the validity of ADAs for pneumonia case identification, an adequate appreciation of this literature is difficult. We systematically reviewed the quality and accuracy of published ADAs to identify adult hospitalized pneumonia cases. Methods We reviewed the Medline, EMBase, and Cochrane Central databases through May 2020. All studies describing ADAs for adult hospitalized pneumonia and at least one accuracy statistic were included. Investigators independently extracted information about the sampling frame, reference standard, ADA composition, and ADA accuracy. Results Thirteen studies involving 24 ADAs were analyzed. Compliance with a 38-item study-quality assessment tool ranged from 17 to 29 (median, 23; interquartile range [IQR], 20 to 25). Study setting, design, and ADA composition varied extensively. Inclusion criteria of most studies selected for high-risk populations and/or increased pneumonia likelihood. Reference standards with explicit criteria (clinical, laboratorial, and/or radiographic) were used in only 4 ADAs. Only 2 ADAs were validated (one internally and one externally). ADA positive predictive values ranged from 35.0 to 96.5% (median, 84.8%; IQR, 65.3 to 89.1%). However, these values are exaggerated for an unselected patient population because pneumonia prevalences in the study cohorts were very high (median, 66%; IQR, 46 to 86%). ADA sensitivities ranged from 31.3 to 97.8% (median, 65.1%; IQR 52.5–72.4). Discussion ADAs for identification of adult pneumonia hospitalizations are highly heterogeneous, poorly validated, and at risk for misclassification bias. Greater standardization in reporting ADA accuracy is required in studies using pneumonia ADA for case identification so that results can be properly interpreted.