PACS-integrated machine learning breast density classifier: clinical validation

To test the performance of a novel machine learning-based breast density tool. The tool utilizes a convolutional neural network to predict the BI-RADS based density assessment of a study. The clinical density assessments of 33,000 mammographic examinations (164,000 images) from one academic medical center (Site A) were used for training. This was an IRB approved HIPAA compliant study performed at two academic medical centers. The validation data set was composed of 500 studies from one site (Site A) and 700 from another (Site B). At Site A, each study was assessed by three breast radiologists and the majority (consensus) assessment was used as truth. At Site B, if the tool agreed with the clinical reading, then it was considered to have correctly predicted the clinical reading. In cases where the tool and the clinical reading disagreed, then the study was evaluated by three radiologists and the consensus reading was used as the clinical reading. For the classification into the four categories of the Breast Imaging Reporting and Data System (BI-RADS®), the AI classifier had an accuracy of 84.6% at Site A and 89.7% at Site B. For binary classification (dense vs. non-dense), the AI classifier had an accuracy of 94.4% at Site A and 97.4% at Site B. In no case did the classifier disagree with the consensus reading by more than one category. The automated breast density tool showed high agreement with radiologists' assessments of breast density. •An automated breast density tool performed with high overall accuracy in the range of 85-90%.•The tool was more likely to agree with a consensus reading than was the original clinical reading.•The tool would improve the reproducibility of density assessments and could be used for triaging to supplemental screening.