Daily peer review of abnormal cervical smears in the assessment of individual practice as an additional method of internal quality control

Objective An important internal quality control system used in the Cancer Prevention and Research Institute cytopathology laboratory in Florence is the peer review procedure, based on the review of all abnormal cytological smears which routinely emerge. Peer review is an important training opportunity for all cytologists, especially for those with less experience. This article shows the results of the peer review procedure. Methods Of the 63 754 Papanicolaou (Pap) smears screened in 2011, 1086 were considered to be abnormal [at least atypical squamous cells of undetermined significance (ASC‐US+)] on primary screening (selected by a single cytologist) and were subjected to the peer review procedure. The overall performance of the laboratory's cytologists was evaluated using a multiple rater analysis and the comparison of each cytologist with the final diagnosis. Further, the agreement was assessed by means of Cohen's kappa and weighted kappa statistics. Results In general, a moderate/substantial level of agreement between the ten cytologists and the final diagnoses was evident. Kappa values for each reader compared with the final diagnosis ranged from 0.54 to 0.69. The overall kappa value was 0.62 [95% confidence interval (CI), 0.58–0.66] and overall weighted kappa value was 0.76 (95% CI, 0.74–0.79). The category‐specific agreement showed the lowest values for atypical squamous cells, cannot exclude high‐grade squamous intraepithelial lesion (ASC‐H). Conclusion In summary, peer review represents an important internal quality control in the evaluation and improvement of inter‐observer agreement and of the functioning of the laboratory as a whole. Multi‐head microscope sessions may improve particularly the reproducibility of borderline diagnoses and, above all, can be an important training contribution for cytologists.