Improving Accuracy in a Computerized Immunization Registry

Background.— Boston's Department of Health and Hospitals established the Boston Immunization Information System (BIIS) as part of a citywide effort to raise immunization rates. Objectives.— Our objectives were 1) to assess the validity of data entry in one computerized immunization database within this system; 2) to identify the types of errors made in data entry; and 3) to assess the effectiveness of an intervention to improve the accuracy of information in this database. Methods.— Chart records were used as the gold standard in comparison with the computerized BIIS database. Children were considered up to date for immunizations if they had received 4 DTPs, 3 polios, and 1 MMR by their second birthday. In March of 1995, billing records were used to identify all children born between March 1, 1992, and September 1, 1994, in one urban health center. These children were between 6 months and 36 months of age at the time. We compared the computer record with the chart, looking for and correcting errors such as incorrect dates and missing immunization data. An intervention was then begun, including a system for reviewing the accuracy of the computerized data at all well-child visits. In October of 1996, the chart review was repeated on children born between March 1, 1992, and September 1, 1995. This age range included all the children in the original review plus those born in the subsequent 12 months. Immunization rates before and after chart reviews were compared for children 24 to 36 months of age. Results.— We initially reviewed 737 of 739 charts (99%). In the follow-up time period, we examined 881 of 943 charts (93%). During the first review, 333 of the 563 (59%) records contained at least one error in data entry, compared with 116 of 646 (18%) in the second review ( P < .0001). During the second review, we also examined the type of errors. Thirty-eight percent of all errors represented vaccines that had not been entered into the computer. Before the study period, analysis of the computerized immunization record showed an immunization up-to-date rate of 24%. This increased to 41% after the initial chart review and to 75% after the intervention period ( P < .0001). Conclusion.— Errors in data entry caused underestimates of immunization rates. Eliminating the errors increased immunization rates immediately through more accurate bookkeeping. Eighteen months later, immunization rates had continued to improve dramatically.