Tunable topological states in layered magnetic materials of MnSb2Te4, MnBi2Se4, and MnSb2Se4

The recently discovered magnetic topological insulator of MnBi2Te4, has been successfully used to explore emerging physical phenomena, such as the quantum anomalous Hall effect (QAHE) and axion insulator state. Based on first-principles calculations, we have systematically investigated the electronic, magnetic, and topological properties of layered MnSb2Te4, MnBi2Se4, and MnSb2Se4, where those materials host similar crystal structure as the MnBi2Te4. Our calculations first show that each bulk system with antiferromagnetic order can be converted into a three-dimensional topologically nontrivial insulator by applying appropriate pressure. Then, we find that ferromagnetic (FM) MnSb2Te4 is a type II Weyl semimetal with a single pair of Weyl points near Fermi level. Specifically, FM MnBi2Se4 and MnSb2Se4 can be readily converted into Weyl semimetals under pressure. Furthermore, we notice that QAHE can also be achieved in layered FM MnSb2Te4 and MnBi2Se4. All those findings demonstrate that the MnBi2Te4-like materials of MnSb2Te4, MnBi2Se4, and MnSb2Se4 are promising candidates to explore intriguing topological quantum states.