Analysis and Suppression for Critical Optical Nonorthogonal Noise in SERF Comagnetometers

Ultrasensitive atomic comagnetometers have proven to be valuable tools for precise measurement and search for new physics. Besides, comagnetometers have potential for inertial navigation and quantum information. Despite their great potential, sensitivity improvement has been limited by different types of noise. To address this issue, the sources and magnitudes of noises are comprehensively analyzed and a suppression method is proposed for the most critical noise source, optical nonorthogonality. The optical nonorthogonality leads to critical transverse pumping effect, thus transverse light shift, and transverse effective magnetic noise. A direct correlation between the optical nonorthogonality and the electronic spin transverse relaxation rate is found, and a method for suppression of the optical nonorthogonality is proposed accordingly. Finally, the optical nonorthogonal noise is suppressed by 3.1 dB. Besides, an increase of 5.2 dB in the scale factor is achieved. With the combined effect of noise suppression and scale factor enhancement, the resulting inertial rotation sensitivity of 2.4 \times 10^{-8} rad/s/Hz ^{1/2} in the frequency range from 1 to 3 Hz is among the most sensitive comagnetometers, which opens up promising opportunities for new physics measurements, such as the search for ultralight dark matter and the fifth force.