Joint segmentation and classification task via adversarial network: Application to HEp-2 cell images

The recognition of epithelial type 2 (HEp-2) cells has become an important tool for detecting autoimmune diseases diagnosis by using indirect immunofluorescence (IIF) images. For many medical image tasks, the accurate segmentation is considered as the primary step for the classification task, which inspires researchers to solve problems by jointly implementing multiple tasks. For this reason, we devise a hybrid network architecture, which combines the segmentation and classification networks for the final classification of HEp-2 cells, where a multi-task generative adversarial networks (GANs) is employed to produce accurate segmentation masks for improving the latter classification performance. Specifically, the devised generation and discrimination subnetworks establish the GANs architecture for the accurate segmentation masks of HEp-2 cells. Also, the original images are utilized as the conditional input, which are concatenated by the generated masks and the ground-truth to train the discriminator for two tasks: the first task determines whether the generated mask is a ground-truth or not and the other one distinguishes the category of the processed HEp-2 cell image. Furthermore, the ResNet-34 and MobileNetv3 are used as segmentation and classification base network, respectively. We modify the MobileNetv3 structure by adding the channel of the middle outputs, which is called augmented channel MobileNetv3 (ACM-Net). Both the discriminator and classifier share the weights of ACM-Net. The extensive experiments on the public ICPR 2016 task1 dataset show that the proposed hybrid-task based GANs structure can obtain promising segmentation and classification performance via jointly training mode, which achieves a Dice of 97.04% and for segmentation and a prediction accuracy of 98.82% for classification, respectively. •We design a novel hybrid framework, which combines the ResNet-34 and augmented channel MobileNetv3 (ACM-Net) to construct the GANs for joint segmentation and classification of HEp-2 cells.•A multi-scale atrous spatial pyramid pooling (MS-ASPP) is proposed to extract and confuse multi-scale spatial information.•The multiple auxiliary classifiers are built to assist the main classifier to obtain better classification performance.•The weights of discriminator are shared with the classifier so that the parameters of the whole network are greatly reduced.