Transformer based multiple superpixel-instance learning for weakly supervised segmenting lesions of interstitial lung disease

Automatic and accurate segmentation of lesions from high-resolution computed tomography (HRCT) images play a critical role in diagnosis of interstitial lung disease (ILD). Fully supervised methods require considerable expertise annotations, resulting in time-consuming and labor-intensive cost. Contrastively, weakly supervised segmentation (WSS) algorithms could alleviate greatly this issue of medical image annotation. Multiple-instance learning (MIL), a class of weakly supervised learning (WSL), has proven to be effective in image segmentation tasks. However, most existing MIL methods based on patches or pixels lack meaningful edge information of the region of interests (ROIs) and leave out of considering the dependence relations of inter-instances. In this study, we propose a novel Transformer based multiple superpixel-instance learning (MSIL) for pixel-level lesions segmentation of ILD. This method introduces superpixels into a MIL framework, which solves the problem of inconsistency in lesion boundary segmentation caused by using patches as instances. In addition, the proposed method introduces Transformer into the MIL framework to capture global or long-range dependencies and establish the relationship between superpixel instances based on the multi-head self-attention, which solves the shortcoming that instances are independent of each other in MIL. Experimental results on private HRCT images confirm that our proposed method can achieve state-of-the-art (SOTA) performance compared to other WSS algorithms, MIL-based WSS algorithms, CAMs-based classification algorithms (CA) and fully supervised segmentation algorithms (FSS). •We proposed a novel weakly supervised segmentation (WSS) algorithm (ViT-MSIL).•We explored the combination of Transformer, superpixels and MIL for WSS on HRCT images.•Our experimental results show that ViT-MSIL achieves state-of-the-art results.•The ViT-MSIL segments the lesion edge more accurately than the patch-based method.