Observation of three-mode parametric instability

Three-mode parametric interactions occur in triply resonant optomechanical systems: Photons from an optical pump mode are coherently scattered to a high-order mode by mechanical motion of the cavity mirrors, and these modes resonantly interact via radiation pressure force when some conditions are met. They can either pump energy into acoustic modes, leading to parametric instability, or extract mechanical energy, leading to optomechanical cooling. Such effects are predicted to occur in long baseline advanced gravitational-wave detectors, possibly jeopardizing their stable operation. We have demonstrated both three-mode cooling and amplification in two different three-mode optomechanical systems. We report an observation of the three-mode parametric instability in a free-space Fabry-Perot cavity, with ring-up amplitude saturation.