Generative and discriminative model-based approaches to microscopic image restoration and segmentation

Abstract Image processing is one of the most important applications of recent machine learning (ML) technologies. Convolutional neural networks (CNNs), a popular deep learning-based ML architecture, have been developed for image processing applications. However, the application of ML to microscopic images is limited as microscopic images are often 3D/4D, that is, the image sizes can be very large, and the images may suffer from serious noise generated due to optics. In this review, three types of feature reconstruction applications to microscopic images are discussed, which fully utilize the recent advancements in ML technologies. First, multi-frame super-resolution is introduced, based on the formulation of statistical generative model-based techniques such as Bayesian inference. Second, data-driven image restoration is introduced, based on supervised discriminative model-based ML technique. In this application, CNNs are demonstrated to exhibit preferable restoration performance. Third, image segmentation based on data-driven CNNs is introduced. Image segmentation has become immensely popular in object segmentation based on electron microscopy (EM); therefore, we focus on EM image processing.