Factors influencing the performance of optical heterodyne detection system

•A modeling method for performance analysis of heterodyne detection system considering wavefront distortion is proposed in this paper. Results show that the heterodyne efficiency is fixed if the amplitude distribution of the signal light obeys a certain distribution, but the phase mismatches can significantly degrade the system sensitivity.•By modeling three factors that cause wavefront distortion: atmospheric turbulence, speckle effect of rough surface and laser linewidth, the correlation between the distortion degree caused by these three factors and the signal-to-noise ratio of heterodyne detection is quantitatively analyzed.•A heuristic search algorithm is proposed to compensate the phase. The average value of the calculated amplitude is 9.4615 and the average value of heterodyne efficiency is 0.9947. The performance is better than the swarm intelligence optimization algorithm.•A high-speed camera heterodyne system was developed to evaluate the effectiveness of heuristic search algorithm for correcting the spatial phase mismatch in applications involving active heterodyne detection. The experimental results show that the compensation amplitude is increased by 25.66 times. As a holographic detection method, optical heterodyne detection has excellent advantages over direct detection. This paper presents a modeling method for the performance analysis of heterodyne detection systems, including wavefront distortion. By modeling three factors that lead to wavefront distortion: atmospheric turbulence, rough surface speckle effect, and laser linewidth, the correlation between the distortion degree caused by the three factors and the heterodyne detection signal-to-noise ratio is quantitatively analyzed. An active optical heterodyne detection system of an array detector based on a high-speed camera is proposed to evaluate the effectiveness of the proposed heuristic search algorithm for phase compensation. Experimental results show that the amplitude of the algorithm is increased by 25.66 times. It can effectively solve the wavefront distortion phase problem caused by the detection target's rough surface.