Reproducibility of Classification and Correction for Verification Bias as Determinants of Performance of Papanicolaou Smear Cytology in the Screening Setting: Experience from the New Independent States of the Former Soviet Union Cohort Study

To assess impact on interlaboratory agreement of 2 optional modes to convert Papanicolaou classification to Bethesda System (TBS) and demonstrate effect of verification bias correction on test performance in the New Independent States of the former Soviet Union (NIS) cohort study. Conventional Pap smears were classified in NIS laboratories and rescreened in a reference center (RC) using modified Papanicolaou classification. Two optional methods (TBS1, TBS2) were used to convert original classification to TBS. All classifications were tested for interlaboratory agreement and performance indicators with and without verification bias correction. Interlaboratory agreement of original classification was fair with regular kappa, but substantial with weighted kappa. TBS1 option (Class 2 in atypical squamous cells of undetermined significance [ASCUS]) did not remedy this limited reproducibility. When Class 2+ (borderline dyskaryosis) was used as cutoff for ASCUS (TBS2), interlaboratory agreement was upgraded to moderate and almost perfect. Test performance was significantly different between NIS and RC only for TBS1 option with ASCUS cutoff RC showing better sensitivity/specificity balance by area under receiver operating characteristic curve test. In converting traditional classification to TBS, selection of appropriate equivalents to TBS categories has a major impact on interlaboratory agreement and performance indicators.