Learning depth-aware features for indoor scene understanding

Many methods have shown that jointly learning RGB image features and 3D information from RGB-D domain is favorable to the indoor scene semantic segmentation task. However, most of these methods need precise depth map as the input and this seriously limits the application of this task. This paper is based on a convolutional neural network framework which jointly learns semantic and the depth features to eliminate such strong constraint. Additionally, the proposed model effectively combines learned depth features, multi-scale contextual information with the semantic features to generate more representative features. Experimental results show that only taken an RGB image as the input, the proposed model can simultaneously obtain higher accuracy than state-of- the-art approaches on NYU-Dv2 and SUN RGBD datasets.