Shadow Detection and Removal using a Hybrid Approach of CTU-Net and subarea-Based illumination Compensation

The presence of image shadows significantly impacts the practical applications of computer vision tasks in real-world environments. For example, image recognition, and image segmentation. Previous methodologies have disregarded the relationship between shadow and non-shadow regions in images, and inadequately utilizing neighboring pixels for shadow removal. In this paper, we introduce CTU-Net, a network model that leverages the relationship between shadows and non-shadows to detect image shadows and subsequently remove them using surrounding pixels. Within CTU-Net, a convolutional transformer fusion module is proposed to merge local and global features within the image. Subsequently, a module termed the shadow and non-shadow complementary mechanism is proposed, which predicts shadow and non-shadow regions simultaneously based on image features, allowing for better shadow prediction through the complementary nature of these regions. Moreover, the image is partitioned into multiple sub-areas, and shadow removal is achieved by combining texture-similar sub-areas with illumination restoration operators, followed by the reconstruction of shadow boundaries based on surrounding pixels. Experimental evaluations conducted on five publicly available datasets demonstrate the efficacy of our proposed shadow detection architecture, with an average performance improvement of 4.62%, and shadow removal algorithm, with an average improvement of 2.8%, measured in terms of balanced error rate and root mean square error.