Large-gap non-Dirac quantum anomalous Hall effect in nitrided TlSb monolayers on nonmagnetic substrates

Two-dimensional atomic monolayers of heavy elements grown on insulating substrates are promising for creating devices that exploit topological quantum states. Here, based on first-principles calculations and k <middle dot> p model, we systematically investigate the electronic structures and topological properties of the TlSb monolayer epitaxially grown on nonmagnetic BaTe(111) surface, TlSb/BaTe(111). / BaTe(111). We discover a non-Dirac quantum spin Hall gap of 331.0 meV at its r point, induced by strong spin-orbit coupling (SOC) of the Sb and Tl p(x)/p(y) orbitals. Remarkably, nitrogen functionalization introduces ferromagnetic magnetization that breaks time-reversal symmetry and results in fully spin-polarized states, where the spin-up bands precisely degenerate at the Fermi energy with quadratic non-Dirac band dispersions. This spin-polarized non-Dirac bands, combined with SOC, opens a nontrivial gap of 258.0 meV with a significant Rashba effect, enabling the realization of a large-gap non-Dirac quantum anomalous Hall (QAH) effect, which is verified by a calculated Chern number of 1 and the observed topological edge states. We elucidate the underlying mechanism of this QAH state using the k <middle dot> p model. Furthermore, our findings suggest that this approach can be extended to other nonmagnetic substrates such as SrTe and BaSe, demonstrating a versatile strategy to achieve large-gap QAH effects without the need for magnetic doping or specialized substrates.