1305 Agreement and reliability of an automated PD-L1 tumor proportion scoring algorithm in non-small cell lung cancer (NSCLC)

BackgroundImmunotherapy has revolutionized the treatment paradigm of advanced non-small cell lung cancer (NSCLC), opening possibilities for long-term survival outcomes with superior tolerability.1 Eligibility for immune checkpoint inhibitors is set by estimating the tumor proportion score (TPS) in programmed cell death ligand 1 (PD-L1) immunohistochemically stained slides.2 However, high interobserver variability in reporting PD-L1 expression may result in suboptimal treatment decisions.3 We developed HALO PD-L1 Lung AI to support pathologists in quantifying PD-L1 expression in NSCLC samples, with the aim of reducing interobserver variability.MethodsHALO PD-L1 Lung AI was trained using 146,984 expert annotations to identify PD-L1 tumor-positive and tumor-negative cells, within segmented tumor regions. The agreement and reliability of the algorithm’s TPS score with scores obtained from three independent pathologists were analyzed by calculatingthe interobserver percent agreement and intraclass correlation coefficient (ICC) in a cohort of 203 whole slide images stained with the SP263 clone.ResultsPairwise pathologist agreement ranged from 74.9% to 77.3%. Agreement of HALO PD-L1 Lung AI TPS scores with the pathologists’ mode (where at least 2/3 pathologists agreed) was 75.4%. Agreement of HALO PD-L1 Lung AI with the pathologists’ mode at the clinically relevant cut-offs 50% was 0.81 (95% CI 0.75 – 0.88), 0.72 (95% CI 0.64 – 0.79), and 0.70 (95% CI 0.57 – 0.81), respectively. In 15 of the 18 disagreement cases at the 1% cut-off, the algorithm score was within a 1–4% range, showing that although in a categorical scale the cases were in disagreement, the results were close on a continuous scale. In fact, the intraclass correlation coefficient (ICC) between the algorithm and pathologists’ TPS scores was 0.95 (95% CI 0.93 – 0.97) and between the three pathologists was 0.96 (95% CI 0.93 – 0.97).ConclusionsThe percent agreement of HALO PD-L1 Lung AI with the pathologists’ mode is in line with the pairwise agreement between pathologists. On the continuous scale, ICC results show good-to-excellent reliability between the algorithm and pathologists’ TPS scores. Computer-aided diagnostic tools such as HALO PD-L1 Lung AI have the potential to increase consistency in the reported TPS results and ultimately improve treatment decision making.ReferencesJiang M, Liu C, Ding D, Tian H, Yu C. Comparative Efficacy and Safety of Anti-PD-1/PD-L1 for the Treatment o