Robust deep learning optical autofocus system applied to automated multiwell plate single molecule localization microscopy

We presenta robust, long‐range optical autofocus system for microscopy utilizing machine learning. This can be useful for experiments with long image data acquisition times that may be impacted by defocusing resulting from drift of components, for example due to changes in temperature or mechanical drift. It is also useful for automated slide scanning or multiwell plate imaging where the sample(s) to be imaged may not be in the same horizontal plane throughout the image data acquisition. To address the impact of (thermal or mechanical) fluctuations over time in the optical autofocus system itself, we utilize a convolutional neural network (CNN) that is trained over multiple days to account for such fluctuations. To address the trade‐off between axial precision and range of the autofocus, we implement orthogonal optical readouts with separate CNN training data, thereby achieving an accuracy well within the 600 nm depth of field of our 1.3 numerical aperture objective lens over a defocus range of up to approximately +/–100 μm. We characterize the performance of this autofocus system and demonstrate its application to automated multiwell plate single molecule localization microscopy. Lay Description Many microscopy experiments involve extended imaging of samples over timescales from minutes to days, during which the microscope can ‘drift’ out of focus. When imaging at high magnification, the depth of field is of the order of one micron and so the imaging system should keep the sample in the focal plane of the microscope objective lens to this precision. Unfortunately, temperature changes in the laboratory can cause thermal expansion of microscope components that can move the focal plane by more than a micron and such changes can occur on a timescale of minutes. This is a particular issue for super‐resolved microscopy experiments using single molecule localization microscopy (SMLM) techniques, for which 1000s of images are acquired, and for automated imaging of multiple samples in multiwell plates. It is possible to maintain the sample in the focal plane focus position by either automatically moving the sample or adjusting the imaging system, for example by moving the objective lens. This is called ‘autofocus’ and is frequently achieved by reflecting a light beam from the microscope coverslip and measuring its position of beam profile as a function of defocus of the microscope. The correcting adjustment is then usually calculated analytically but there is rece