Distortion Convolution Module for Semantic Segmentation of Panoramic Images Based on the Image-Forming Principle

Semantic segmentation of panoramic images plays a key role in many applications, such as security monitoring and autonomous driving. With the rapid development of deep learning, some deep networks are developed to segment panoramic images semantically. However, these networks don't have the special modules to correct the image distortion of panoramic images according to the distortion principle, which makes feature extraction unreasonable during the convolution because of the image distortion. This article proposes a novel semantic segmentation network for panoramic images of outdoor scenes based on the distortion convolution. The network contains an encoder and a decoder. The encoder consists of a distortion convolutional module (DCM), a residual network, and an atrous spatial pyramid pooling (ASPP). The DCM is developed to correct the image distortion according to the image-forming principle. In the decoder, a deep feature aggregation network (DFAN) is designed to fully fuse low-level features with high-level features. The proposed network introduces the DCM and DFAN into the semantic segmentation of panoramic images, which improves the segmentation accuracy. The experiments demonstrate that the proposed network has good performance for different outdoor scenes and plays an important role in measurement applications.