Anisotropy of the zigzag order in the Kitaev honeycomb magnet $\alpha$-RuBr$_3

Kitaev materials often order magnetically at low temperatures due to the presence of non-Kitaev interactions. Torque magnetometry is a very sensitive technique for probing the magnetic anisotropy, which is critical in understanding the magnetic ground state. In this work, we report detailed single-crystal torque measurements in the proposed Kitaev candidate honeycomb magnet $\alpha$-RuBr$_3$, which displays zigzag order below 34 K. Based on angular-dependent torque studies in magnetic fields up to 16 T rotated in the plane normal to the honeycomb layers, we find an easy-plane anisotropy with a temperature dependence of the torque amplitude following closely the behaviour of the powder magnetic susceptibility. The torque for field rotated in the honeycomb plane has a clear six-fold periodicity with a saw-tooth shape, reflecting the three-fold symmetry of the crystal structure and stabilization of different zigzag domains depending on the field orientation, with a torque amplitude that follows an order parameter form inside the zigzag phase. By comparing experimental data with theoretical calculations we identify the relevant anisotropic interactions and the role of the competition between different zigzag domains in this candidate Kitaev material.