In Data We Trust? Comparison of Electronic Versus Manual Abstraction of Antimicrobial Prescribing Quality Metrics for Hospitalized Veterans With Pneumonia

BACKGROUND:Electronic health records provide the opportunity to assess system-wide quality measures. Veterans Affairs Pharmacy Benefits Management Center for Medication Safety uses medication use evaluation (MUE) through manual review of the electronic health records. OBJECTIVE:To compare an electronic MUE approach versus human/manual review for extraction of antibiotic use (choice and duration) and severity metrics. RESEARCH DESIGN:Retrospective. SUBJECTS:Hospitalizations for uncomplicated pneumonia occurring during 2013 at 30 Veterans Affairs facilities. MEASURES:We compared summary statistics, individual hospitalization-level agreement, facility-level consistency, and patterns of variation between electronic and manual MUE for initial severity, antibiotic choice, daily clinical stability, and antibiotic duration. RESULTS:Among 2004 hospitalizations, electronic and manual abstraction methods showed high individual hospitalization-level agreement for initial severity measures (agreement=86%–98%, κ=0.5–0.82), antibiotic choice (agreement=89%–100%, κ=0.70–0.94), and facility-level consistency for empiric antibiotic choice (anti-MRSA r=0.97, P