KaryoXpert: An accurate chromosome segmentation and classification framework for karyotyping analysis without training with manually labeled metaphase-image mask annotations

Automated karyotyping is of great importance for cytogenetic research, as it speeds up the process for cytogeneticists through incorporating AI-driven automated segmentation and classification techniques. Existing frameworks confront two primary issues: Firstly the necessity for instance-level data annotation with either detection bounding boxes or semantic masks for training, and secondly, its poor robustness particularly when confronted with domain shifts. In this work, we first propose an accurate segmentation framework, namely KaryoXpert. This framework leverages the strengths of both morphology algorithms and deep learning models, allowing for efficient training that breaks the limit for the acquirement of manually labeled ground-truth mask annotations. Additionally, we present an accurate classification model based on metric learning, designed to overcome the challenges posed by inter-class similarity and batch effects. Our framework exhibits state-of-the-art performance with exceptional robustness in both chromosome segmentation and classification. The proposed KaryoXpert framework showcases its capacity for instance-level chromosome segmentation even in the absence of annotated data, offering novel insights into the research for automated chromosome segmentation. The proposed method has been successfully deployed to support clinical karyotype diagnosis. •KaryoXpert can be trained without manually annotated ground truth instance masks.•It merges advantages from both morphology algorithms and deep-learning models.•It performs high accuracy despite domain shifts, batch effects, and class similarity.•GPU acceleration enables real-time inference with cutting-edge clinical accuracy.