Performance of automatic machine learning versus radiologists in the evaluation of endometrium on computed tomography

Purpose In this study, we developed radiomic models that utilize a combination of imaging features and clinical variables to distinguish endometrial cancer (EC) from normal endometrium on routine computed tomography (CT). Methods A total of 926 patients consisting of 416 endometrial cancer (EC) and 510 normal endometrium were included. The CT images of these patients were segmented manually, and divided into training, validation, testing and external testing sets. Non-texture and texture features of these images with endometrium or uterus as region of interest were extracted. The clinical feature “age” was also included in the feature set. Feature selection and machine learning classifier were applied to normalized feature set. This manual optimized combination was then compared with the best pipeline exported by Tree-Based Pipeline Optimization Tool (TPOT) on testing and external testing set. The performances of these machine learning pipelines were compared to that of radiologists. Results The manual expert optimized pipeline using the “reliefF” feature selection method and “Bagging” classifier on the external testing set achieved a test ROC AUC of 0.73, accuracy of 0.73 (95% CI 0.62–0.82), sensitivity of 0.64 (95% CI 0.45–0.79), and specificity of 0.78 (95% CI 0.65–0.87), while TPOT achieved a test ROC AUC of 0.79, accuracy of 0.80 (95% CI 0.70–0.87), sensitivity of 0.61 (95% CI 0.43–0.77), and specificity of 0.90 (95% CI 0.78–0.96). When compared to average radiologist performance, the TPOT achieved higher test accuracy (0.80 vs. 0.49, p