Generation of NIR Spectral Band from RGB Image with Wavelet Domain Spectral Extrapolation Generative Adversarial Network

Near-infrared (NIR) imaging exhibits outstanding penetration capabilities and robust anti-interference characteristics. However, acquiring high-resolution and high-fidelity NIR images is difficult due to the limited mobility, high cost, and low resolution of NIR imaging hardware. In this paper, we proposed a new end-to-end Wavelet Domain Spectral Extrapolation Generative Adversarial Network (WSEGAN) to generate highly realistic NIR images from RGB images. Since RGB and NIR images have different types of noise and artifacts, which affects the quality of NIR images generation. We design a generator with a discrete wavelet transform and an attention mechanism to capture multi-resolution contextual information, and a multi-scale discriminator to capture detailed and global features. Then, the proposed approach is evaluated by three different datasets to achieve optimal results regarding both visual effects and quantitative evaluation. The normalized difference vegetation index (NDVI) is utilized to validate the effectiveness of the generated NIR images. The result demonstrates a strong correlation between the generated images and the actual vegetation distribution. More importantly, the proposed network is testified to generate NIR images to be fused with the RGB image source for agricultural target detection tasks. In dark conditions, results show that using multi-modal data instead of RGB images improves mAP0.5 detection accuracy by 4% on the CAPSICUM dataset and by 8% on the KIWI dataset across five object detection methods. This follows the physical significance of the NIR imaging and demonstrates the potential use of the NIR images generated by the proposed method.