Optimal phase mask design for coherent modulation imaging by deep learning

•A joint optimization between phase mask design and phase retrieval based on CMI and deep learning.•An accurate physical layer is introduced to generate sufficiently extensive datasets, ensuring the robustness of the model.•The optimal phase mask and results are obtained simultaneously by training a neural network to retrieve phase information and updating the phase mask through gradient descent.•Efficient ambiguity removal and improvement of image quality are achieved by the utilization of deep learning and co-optimization.•Time required for reconstruction can be reduced without iterative projection. Since detectors can only record the intensity of the wavefield, it is a long-standing issue to find an effective method to retrieve the phase information. A classic solution to the problem is coherent diffractive imaging (CDI), a lensless phase retrieval technique that reconstructs a complex wavefield from diffraction intensity data recorded by detectors using iterative algorithms. Coherent modulation imaging (CMI), an extended approach to CDI, enhances the process by introducing a known modulation to the wavefield at the sample plane, which removes ambiguities of CDI and reconstructs a complex wavefield from a single diffraction pattern. CMI relaxes dynamic range requirements for detectors and exhibits faster convergence and higher reliability. However, it's important to note that different modulation patterns can impact the results of CMI. Here we propose a co-optimization between optimal phase mask design and phase retrieval based on deep learning. By incorporating an accurate physical layer to simulate forward propagation, training a neural network to retrieve phase information and adding a module to optimize the phase mask, we effectively remove the ambiguities of the phase retrieval problem and improve reconstruction accuracy. Numerical simulations and experiments demonstrate that our method produces high-quality and robust reconstructions. In addition, instead of relying on iterative algorithms, the implementation of deep learning combined with physical layer can effectively accelerate the reconstruction process.