FMCA-Net: A feature secondary multiplexing and dilated convolutional attention polyp segmentation network based on pyramid vision transformer

Polyp segmentation is of great significance in diagnosing and treating related symptoms. Existing polyp segmentation methods have performed well in solving the problems of intra-polyp inconsistency and inter-polyp distinguishability. However, three shortcomings still exist: (1) The decoder does not fully use the initially extracted polyp features. (2) The segmentation edges are fuzzy, and the boundaries are unclear. (3) The network structure is becoming increasingly complex and needs to be clarified. We propose a feature secondary reuse and hole convolutional attention network (FMCA-Net) based on a Pyramid Vision Transformer to solve these problems. Firstly, we propose a feature secondary reuse module (D-BFRM) to process the polyp features of different scales initially extracted in the encoder. After two stages of reuse processing, they are used as references for the remaining branches. This way, feature information such as polyp size, shape, and number can be fully obtained, ensuring the model’s fitting ability. Secondly, we also propose a dilated convolutional attention module group (DCBA&DCGA), in which DCBA is used to process each branch’s features further. In contrast, DCGA processes the final global features to distinguish the boundaries between polyps and backgrounds further and improve the model’s overall generalization ability. Finally, we use the idea of modularization in the model to make the structure more concise and clear. We objectively evaluate the proposed method on five public polyp segmentation datasets. The experimental results show that FMCANet has excellent learning and fitting ability and strong generalization ability. At the same time, the idea of modularization also has obvious advantages in the simplicity and clarity of the model structure.