Nanomechanically-induced nonequilibrium quantum phase transition to a self-organized density wave of a Bose-Einstein condensate

We report on a nonequilibrium quantum phase transition (NQPT) in a hybrid quantum many-body system consisting of a vibrational mode of a damped nanomembrane interacting optomechanically with a cavity, whose output light couples to two internal states of an ultracold Bose gas held in an external quasi-one-dimensional box potential. For small effective membrane-atom couplings, the system is in a homogeneous Bose-Einstein condensate (BEC) steady state, with no membrane displacement. Depending on the transition frequency between the two internal atomic states, either one or both internal states are occupied. By increasing the atom-membrane couplings, the system transitions to a symmetry-broken self-organized BEC phase, which is characterized by a considerably displaced membrane steady-state and density-wave-like BEC profiles. This NQPT can be both discontinuous and continuous for a certain interval of transition frequencies and is purely discontinuous outside of it.