Reliability and accuracy of the screening for adverse events in Brazilian hospitals

Objective. To analyze the reliability and accuracy of the screening for adverse events (AEs) conducted by nurses taking the assessment by medical residents as the reference. Design. A validation study of the screening phase of a previous retrospective cohort study based on the patient record review that estimated the incidence of AEs (base study). Setting. Three general teaching hospitals in the State of Rio de Janeiro, Brazil. Participants. A subsample of 242 medical records randomly selected from an original sample of 1103 previously evaluated records. Main Outcome Measure(s). A two-step approach was used for analysis: the identification of at least one screening criterion (first endpoint) and the validation of each identified criterion (second endpoint), taking the assessment by medical residents as the reference. Kappa coefficient; simple percentage agreement; sensitivity; specificity; positive and negative predictive values were calculated. Results. The total agreement between medical residents and nurses on the presence of screening criteria was moderate (78.9%, K=0.55). Specificity (81.6%) was higher than sensitivity (74.4%). Nurses detected more screening criteria that were later confirmed as true AEs (179 vs. 171, respectively). Significant differences in the detection of the screening criteria: Other complications', 'Hospitalization injury' and 'Prior admission' were observed. Conclusion. The results suggested a good performance of the nurses in the screening for detection of AE and showed significant differences in relation to detection of specific screening criteria among reviewers. A better understanding of the screening process and the performance of reviewers was provided.