Image and video spatial super-resolution via bandlet-based sparsity regularization and structure tensor

A new framework for single image and video super-resolution is proposed in this paper. The main idea is to employ the geometric details of the image in a way that minimizes the possible artifacts caused by the super-resolution process. To this end, we benefit from the bandlet transform which captures the image surface geometry quite effectively. The proposed single image super-resolution method is a two-stage scheme. In the first step, edges and high frequency details of the image are interpolated in the enlarged image. This interpolation uses a structure tensor, which is modified according to the geometry layer revealed by the image bandlets. In the second stage, the edge interpolated image is fed to an inpainting scheme to estimate the pixel values for the new spatial locations within the enlarged image. These locations are treated as missing pixels and then a precise regularization in the bandlet domain is performed to fill-in the missing pixels. In a bid to avoid flickering artifacts after frame-by-frame spatially super-resolving videos, a pixel intensity refinement stage is added to the procedure which takes into account the motion flows of the frames. Due to the use of geometric details that bandlet transform captures and the edge interpolation stage, the method results in high quality, high resolution images with no over-smoothing or blurring effect while in case of video sequences it also preserves temporal consistency. Our experimental results demonstrate the effectiveness of the proposed super-resolution technique for both still images and videos.