Kitaev-Heisenberg model on a honeycomb lattice: possible exotic phases in iridium oxides A2IrO3

We derive and study a spin one-half Hamiltonian on a honeycomb lattice describing the exchange interactions between Ir4+ ions in a family of layered iridates A2IrO3 (A=Li,Na). Depending on the microscopic parameters, the Hamiltonian interpolates between the Heisenberg and exactly solvable Kitaev models. Exact diagonalization and a complementary spin-wave analysis reveal the presence of an extended spin-liquid phase near the Kitaev limit and a conventional Néel state close to the Heisenberg limit. The two phases are separated by an unusual stripy antiferromagnetic state, which is the exact ground state of the model at the midpoint between two limits.