Breast tumor segmentation in digital mammograms using spiculated regions

•A new method is designed for mammography tumor segmentation.•The performance of CAD mammography systems relies on segmentation accuracy.•The proposed segmentation method extracts the spiculated regions and tumor core.•Spiculated part is one of the significant characteristics of malignant breast tumors.•The pixels of a spiculated region are located along one line. Mammogram image segmentation is the process of partitioning mammograms into meaningful and separate areas. However, during the segmentation process, masses are extracted and the spiculated regions of a mass, which contain significant characteristics of the mass margins, are omitted. The present research introduces a new method for segmentation of tumor mammograms that extracts the spiculated regions and the mass core. Generally, the pixels of a spiculated region are located along a line and the pixels of the mass core regions are similar. The proposed method extracts these regions using the differences between a pixel and its adjacent pixels. The proposed method uses three thresholds to delete redundant pixels from the spiculated regions and the mass core. These regions then are merged to form the segmented tumor. The results show that the respective mean of the Dice and Jaccard coefficients for the suggested segmentation method, respectively, are 0.9309 and 0.9024 for MIAS and 0.9557 and 0.9132 for DDSM. Quantitative analysis of the results confirms that the suggested segmentation method is comparable to other techniques and extracts the segmentation of tumor accurately.