Microscopic vision system with all-in-focus and depth images

A high-speed digital processed microscopic observational system for telemicrooperation is proposed with a dynamic focusing system and a high-speed digital-processing system using the "depth from focus" criterion. In our previous work, we proposed a system that could simultaneously obtain an "all-in-focus image" as well as the "depth" of an object. In reality, in a microoperation, it is not easy to obtain good visibility of objects with a microscope focused at a shallow depth, especially in microsurgery and DNA studies, among other procedures. In this sense, the all-in-focus image, which keeps an in-focus texture over the entire object, is useful for observing microenvironments with the microscope. However, one drawback of the all-in-focus image is that there is no information about the objects' depth. It is also important to obtain a depth map and show the 3D microenvironments at any view angle in real time to actuate the microobjects. Our earlier system with a dynamic focusing lens and a smart sensor could obtain the all-in-focus image and the depth in 2 s. To realize real-time microoperation, a system that could process at least 30 frames per second (60 times faster than the previous system) would be required. This paper briefly reviews the depth from focus criterion to Simultaneously achieve the all-in-focus image and the reconstruction of 3D microenvironments. After discussing the problem inherent in our earlier system, a frame-rate system constructed with a high-speed video camera and FPGA (field programmable gate array) hardware is discussed. To adapt this system for use with the microscope, new criteria to solve the "ghost problem" in reconstructing the all-in-focus image are proposed. Finally, microobservation shows the validity of this system.