Early effect of validation efforts of Massachusetts syndromic surveillance data

ObjectiveTo develop a detailed data validation strategy for facilitiessending emergency department data to the Massachusetts SyndromicSurveillance program and to evaluate the validation strategy bycomparing data quality metrics before and after implementation ofthe strategy.IntroductionAs a participant in the National Syndromic Surveillance Program(NSSP), the Massachusetts Department of Public Health (MDPH)has worked closely with our statewide Health Information Exchange(HIE) and National Syndromic Surveillance Program (NSSP)technical staff to collect and transmit emergency department (ED)data from eligible hospitals (EHs) to the NSSP. Our goal is to ensurecomplete and accurate data using a multi-step process beginning withpre-production data and continuing after EHs are sending live datato production.MethodsWe used an iterative process to establish a framework formonitoring data quality during onboarding of EHs into our syndromicsurveillance system and kept notes of the process.To evaluate the framework, we compared data received duringthe month of January 2016 to the most recent full month of data(June 2016) to describe the following primary data quality metricsand their change over time: total and daily average of message andvisit volume; percent of visits with a chief complaint or diagnosiscode received in the NSSP dataset; and percentage of visits with achief complaint/diagnosis code received within a specified time ofadmission to the ED.ResultsThe strategies for validation we found effective includedexamination of pre-production test HL7 messages and the executionof R scripts for validation of live data in the staging and productionenvironments. Both the staging and production validations areperformed at the individual message level as well as the aggregatedvisit level, and included measures of completeness for requiredfields (Chief Complaint, Diagnosis Codes, Discharge Dispositions),timeliness, examples of text fields (Chief Complaint and TriageNotes), and demographic information. We required EHs to passvalidation in the staging environment before granting access to senddata to the production environment.From January to June 2016, the number of EHs sending data tothe production environment increased from 44 to 48, and the numberof messages and visits captured in the production environmentincreased substantially (see Table 1). The percentage of visits witha chief complaint remained consistently high (>99%); howeverthe percentage of visits with a chie