Automated tibiofemoral joint segmentation based on deeply supervised 2D-3D ensemble U-Net: Data from the Osteoarthritis Initiative

Improving longevity is one of the greatest achievements in humanity. Because of this, the population is growing older, and the ubiquity of knee osteoarthritis (OA) is on the rise. Nonetheless, the understanding and ability to investigate potential precursors of knee OA have been impeded by time-consuming and laborious manual delineation processes which are prone to poor reproducibility. A method for automatic segmentation of the tibiofemoral joint using magnetic resonance imaging (MRI) is presented in this work. The proposed method utilizes a deeply supervised 2D-3D ensemble U-Net, which consists of foreground class oversampling, deep supervision loss branches, and Gaussian weighted softmax score aggregation. It was designed, optimized, and tested on 507 3D double echo steady-state (DESS) MR volumes using a two-fold cross-validation approach. A state-of-the-art segmentation accuracy measured as Dice similarity coefficient (DSC) for the femur bone (98.6 ± 0.27%), tibia bone (98.8 ± 0.31%), femoral cartilage (90.3 ± 2.89%), and tibial cartilage (86.7 ± 4.07%) is achieved. Notably, the proposed method yields sub-voxel accuracy for an average symmetric surface distance (ASD) less than 0.36 mm. The model performance is not affected by the severity of radiographic osteoarthritis (rOA) grades or the presence of pathophysiological changes. The proposed method offers an accurate segmentation with high time efficiency (~62 s) per 3D volume, which is well suited for efficient processing and analysis of the large prospective cohorts of the Osteoarthritis Initiative (OAI). •We presented a robust automatic tibiofemoral joint segmentation framework based on deeply supervised 2D-3D ensemble U-Net•Foreground oversampling is performed to combat class imbalance - one-third of the patch contains at least one foreground class•Deep supervision (weighted auxiliary loss) is considered a form of regularization to alleviate the gradient vanishing problem•The end-to-end framework provides an exceptional segmentation speed leadership (~ 62 seconds) per 3D volume