M3 Net: A multi-scale multi-view framework for multi-phase pancreas segmentation based on cross-phase non-local attention

The complementation of arterial and venous phases visual information of CTs can help better distinguish the pancreas from its surrounding structures. However, the exploration of cross-phase contextual information is still under research in computer-aided pancreas segmentation. This paper presents M3 Net, a framework that integrates multi-scale multi-view information for multi-phase pancreas segmentation. The core of M3 Net is built upon a dual-path network in which individual branches are set up for two phases. Cross-phase interactive connections bridging the two branches are introduced to interleave and integrate dual-phase complementary visual information. Besides, we further devise two types of non-local attention modules to enhance the high-level feature representation across phases. First, we design a location attention module to generate cross-phase reliable feature correlations to suppress the misalignment regions. Second, the depth-wise attention module is used to capture the channel dependencies and then strengthen feature representations. The experiment data consists of 224 internal CTs (106 normal and 118 abnormal) with 1 mm slice thickness, and 66 external CTs (29 normal and 37 abnormal) with 5 mm slice thickness. We achieve new state-of-the-art performance with average DSC of 91.19% on internal data, and promising result with average DSC of 86.34% on external data.