Depth Map Restoration from Under-sampled Data

Depth map sensed by low-cost active sensors are often limited in resolution, whereas depth information achieved from structure from motion or sparse depth scanning techniques may result in a sparse point cloud. Achieving a high resolution (HR) depth map from a low resolution (LR) depth map or densely reconstructing a sparse non-uniformly sampled depth map are fundamentally similar problems with dierent types of up-sampling requirements. The first problem involves upsampling in a uniform grid, whereas the second type of problem requires an up-sampling in a non-uniform grid. In this paper, we propose a new approach to address such issues in a unified framework, based on sparse representation. Unlike, most of the approaches of depth map restoration, our approach does not require an HR intensity image. Based on example depth maps, a dictionary of exemplars is constructed, and is used to restore HR/dense depth map. In case of uniform up-sampling of LR depth map, an edge preserving constraint is used for preserving the discontinuity present in the depth map, and a pyramidal reconstruction strategy is applied in order to deal with higher up-sampling factors. For up-sampling of non-uniformly sampled sparse depth maps, we compute the missing information in local patches from that from similar exemplars. Furthermore, we also suggest an alternative method of reconstructing dense depth map from very sparse non-uniformly sampled depth data by sequential cascading of uniform and non-uniform up-sampling techniques. We provide a variety of qualitative and quantitative results to demonstrate the ecacy of our approach for depth map restoration.