Generative data augmentation by conditional inpainting for multi-class object detection in infrared images

Multi-class object detection in infrared images is important in military and civilian use. Deep learning methods can obtain high accuracy but require a large-scale dataset. We propose a generative data augmentation framework DOCI-GAN, for infrared multi-class object detection with limited data. Contributions of this paper are four-folds. Firstly, DOCI-GAN is designed as a conditional image inpainting framework, yielding paired infrared multi-class object image and annotation. Secondly, a text-to-image converter is formulated to transform text-format object annotations to bounding box mask images, leading the augmentation to be mask-image-to-raw-image translation. Thirdly, a multiscale morphological erosion-based loss is created to alleviate the intensity inconsistency between inpainted local backgrounds and global background. Finally, for generating diverse images, artificial multi-class object annotations are integrated with real ones during augmentation. Experimental results demonstrated that DOCI-GAN augments dataset with high-quality infrared multi-class object images, consequently improving the accuracy of object detection baselines. •A data augmentation method DOCI-GAN for infrared multi-class object detection.•DOCI-GAN is designed as a conditional image inpainting framework.•Based on artificial bounding box masks, DOCI-GAN yields paired object image.•DOCI-GAN is capable of generating high-quality infrared multi-class object images.•DOCI-GAN is also capable of generating infrared single-class object images.•Datasets augmented by DOCI-GAN help to improve object detection accuracy of models.