A Texture Integrated Deep Neural Network for Semantic Segmentation of Urban Meshes

Three-dimensional (3D) geo-information is essential for many urban related applications. Point cloud and mesh are two common representations of the 3D urban surface. Compared to point cloud data, mesh possesses indispensable advantages, such as high-resolution image texture and sharp geometry representation. Semantic segmentation, as an important way to obtain 3D geo-information, however, is mainly performed on the point cloud data. Due to the complex geometry representation and lack of efficient utilizing of image texture information, the semantic segmentation of the mesh is still a challenging task for urban 3D geo-information acquisition. In this paper, we propose a texture and geometry integrated deep learning method for the mesh segmentation task. A novel texture convolution module is introduced to capture image texture features. The texture features are concatenate with non-texture features on a point cloud that represents by the center of gravity (COG) of the mesh triangles. A hierarchical deep network is employed to segment the COG point cloud. Our experimental results show that the proposed network significantly improves the accuracy with the introduced texture convolution module (1.9% for overall accuracy and 4.0% for average F1 score). It also compares with other state-of-the-art methods on the public SUM-Helsinki dataset and achieves considerable results.