Optical flow driven interpolation for isotropic FIB-SEM reconstructions

Background and Objective: Focused Ion Beam - Scanning Electron Microscopy (FIB-SEM) allows three-dimensional ultrastructural analysis of cells and tissues at the nanoscale. The technique iteratively removes a section of the sample with a FIB and takes an SEM image from the exposed surface. The section thickness is usually higher than the image pixel size to reduce acquisition time, thus resulting in anisotropic resolution. In this work, we explore novel interpolation methods along the sectioning direction to produce isotropic resolution and facilitate proper interpretation of the FIB-SEM 3D volumes. Methods: Classical interpolation methods are usually applied in this context under the assumption that the changes through successive images are relatively smooth. However, the actual 3D arrangement of the structures in the sample may cause significant changes in the biological features between consecutive images of the FIB-SEM stacks. We have developed a novel interpolation strategy that accounts for this variation by using the Optical Flow (OF) to estimate it. As an intermediate stage, OF-compensated images are produced by aligning the spatial regions of the biological structures. Interpolated images are then generated from these OF-compensated images. The final isotropic stack is assembled by interleaving the interpolated images with the original images of the anisotropic stack. Results: OF-driven and classical interpolation methods were compared using an objective assessment based on Pearson Correlation Coefficient (PCC) and a qualitative evaluation based on visual results, using public datasets and representative anisotropy conditions. The objective assessment demonstrated that the OF-driven interpolation always yields higher PCC values, with interpolated images closer to the ground truth. The qualitative evaluation corroborated those results and confirmed that classical interpolation may blur areas with substantial changes between consecutive images whereas OF-driven interpolation provides sharpness. Conclusions: We have developed an OF-driven interpolation approach to generating FIB-SEM stacks with isotropic resolution from experimental anisotropic data. It adapts to the rapid variation of the biological structures observed through the images of the FIB-SEM stack. Our approach outperforms classical interpolation and manages to produce sharp interpolated views in cases where there are significant changes between consecutive experimental images.