MixingMask: A contour-aware approach for joint object detection and instance segmentation

Remarkable achievements have been made in object detection and segmentation tasks. However, there remains a noticeable scarcity of methodologies that can achieve satisfactory results in both tasks simultaneously. To address this problem, we present a solution called MixingMask, in which we have a key insight to provide specific attention to boundary features by leveraging contour-based segmentation methods. Specifically, our approach commences with a novel contour deformation module that employs mixing operations coupled with the proposed adaptive feature sampling. Successively, the contours are encoded using a decoupled vector for bounding box regression, thereby effectively associating the contour shape with its scale and position. Lastly, we incorporate the proposed contour regression module into the baseline method to achieve specialized attention to boundary features. Such a design not only successfully remedies the prevailing disregard towards boundary features but also forms an implicit liaison between object detection and instance segmentation tasks. Comprehensive experimental assessments validate the superior performance of the proposed method in both object detection and instance segmentation tasks. •We propose Adaptive Global Deformation, a contour regression method using learnable sampling.•We use contour vertices as intermediates for boundary attention, bridging the gap between detection and segmentation.•The proposed method achieves promising performance on COCO datasets in detection and segmentation both.