Global phase diagram, possible chiral spin liquid, and topological superconductivity in the triangular Kitaev-Heisenberg model

The possible ground states of the undoped and doped Kitaev-Heisenberg model on a triangular lattice are studied. For the undoped system, a combination of the numerical exact diagonalization calculation and the four-sublattice transformation analysis suggests one possible exotic phase and four magnetically ordered phases, including a collinear stripe pattern and a noncollinear spiral pattern in the global phase diagram. The exotic phase near the antiferromagnetic (AF) Kitaev point is further investigated using the Schwinger-fermion mean-field method, and we obtain an energetically favorable Z(2) chiral spin liquid with a Chern number +/- 2 as a promising candidate. At finite doping, we find that the AF Heisenberg coupling supports an s-wave or a d(x2-y2) + id(xy)-wave superconductivity (SC), while the AF and the ferromagnetic Kitaev interactions favor a d(x2-y2) + id(xy)-wave SC and a time-reversal invariant topological p-wave SC, respectively. Possible experimental realizations and related candidate materials are also discussed.