Breast nodule classification with two-dimensional ultrasound using Mask-RCNN ensemble aggregation

•Breast nodule classification (benign vs. malignant) on two-dimensional ultrasound images initially marked as BI-RADS 3 and 4 yields an area under the receiver operating characteristic curve of 0.67.•A mask region-based convolutional neural network can be implemented to solve the classification problem.•Despite high variability in original ultrasound images, the neural network can discriminate between characteristics based solely on image features. The purpose of this study was to create a deep learning algorithm to infer the benign or malignant nature of breast nodules using two-dimensional B-mode ultrasound data initially marked as BI-RADS 3 and 4. An ensemble of mask region-based convolutional neural networks (Mask-RCNN) combining nodule segmentation and classification were trained to explicitly localize the nodule and generate a probability of the nodule to be malignant on two-dimensional B-mode ultrasound. These probabilities were aggregated at test time to produce final results. Resulting inferences were assessed using area under the curve (AUC). A total of 460 ultrasound images of breast nodules classified as BI-RADS 3 or 4 were included. There were 295 benign and 165 malignant breast nodules used for training and validation, and another 137 breast nodules images used for testing. As a part of the challenge, the distribution of benign and malignant breast nodules in the test database remained unknown. The obtained AUC was 0.69 (95% CI: 0.57–0.82) on the training set and 0.67 on the test set. The proposed deep learning solution helps classify benign and malignant breast nodules based solely on two-dimensional ultrasound images initially marked as BIRADS 3 and 4.