P-269 A validation study for artificial intelligence (AI) compared with manual annotation, using donor eggs reveals that AI accurately predicts blastulation

Abstract Study question Are the annotations produced by AI comparable to manual annotations? Does AI accurately assess fertilisation checks, and predict embryo usage and blastulation compared to embryologists? Summary answer Automatic annotations by AI was consistent with manual annotations. AI implantation algorithms had strong prediction of blastulation and embryo usage. What is known already Currently, embryos are manually annotated for specific morphokinetic features during embryo development. This is a labour-intensive process, and dependent on training and experience, leading to inter and intra clinic variation. The decision to transfer, freeze or discard embryos relies heavily on these annotations. It is paramount that we develop a tool that will provide consistency and accuracy in annotation and produce scores that can facilitate decisions around embryo usage. AI has demonstrated its potential to achieve this, but first must be validated before its integration into clinical practice. There have been no such studies demonstrating this so far. Study design, size, duration Retrospective cohort study, that took place between September to December 2021 at a private fertility clinic in Spain. To control for embryo variability, this study only included 179 time-lapse videos for embryos created from donor eggs. This was based on the understanding that donor eggs are more likely to produce better quality blastocysts and embryos and thus will give the most optimal conditions for annotation in a validation framework. Participants/materials, setting, methods The same time-lapse cultured embryos were annotated manually and automatically by CHLOE(Fairtility, an AI-based tool). Manual and CHLOE annotations were compared to assess the strength of agreement (i) using intra-class correlation (ICC), and (ii) the proportion of corrections required at the pronuclei (PN) stage. AI accuracy in predicting blastulation at 30hours, and blastulation before 116 hours, was also assessed using AUC as the efficacy metric. Embryo usage was compared with the AI-generated ranking of embryos. Main results and the role of chance The majority of morphokinetic variables showed a very-strong agreement, with an ICC range of (0.81-1.00), namely for; tPNf, t2, t3, t5, t7, tSB, tB and tEB. Only t4 (0.5) showed a moderate agreement. On average (Mean+-Standard deviation), AI annotated t4 later than embryologists (36+-5vs39+-10 (hours)). All other variables fell within a strong ICC of (0.61-0.