RestoreNet: a deep learning framework for image restoration in optical synthetic aperture imaging system

•Non-blind algorithms, such as the Wiener filter and the Lucy-Richardson algorithm, are always used to restore images of the optical synthetic aperture imaging (OSAI) system, but can't apply in practice easily due to the unknown prior information of system. Deep learning is a new technology rising nowadays, and has many successful application cases in different fields. An improved U-shaped network named RestoreNet is designed to remove the blur and recover the image from OSAI system.•After enough training, RestoreNet has advantages as below:•Restoring ability: After enough training, RestoreNet can remove the blur definitely and easily in OSAI system;•Fast for applying: In testing period, the recovering time for one image is just ~30ms, but the restoration effect is better than the Lucy-Richardson algorithm, which takes 470ms for one image;•Prior information unknown: RestoreNet can restore images without the prior information of system, for example, the point spread function or the optical transfer function, which is not provided by the non-blind algorithms;•Generalization ability: RestoreNet has strong ability in generalization ability whether facing different types of images or imaging systems with different sub-aperture arrangements. Imaging blur is an inevitable problem in optical synthetic aperture imaging system because of the low response of frequency. Non-blind deconvolution algorithms are usually used for image restoration to obtain clear and high-resolution images. However, accurate prior information on the optical transfer function of system is required in the non-blind methods. As a data-driven approach, recent developments in deep learning have shown great potential in image processing. In this paper, a U-shaped deep learning framework named RestoreNet is proposed for image restoration, especially for removing the blur of optical synthetic aperture imaging system in a blind way. Numerical simulation and experiment results show that RestoreNet is an effective alternative with great restoration ability, stability and generalization in the image restoration of optical synthetic aperture imaging system.