The competition between the intrinsic and Rashba spin-orbit coupling and effects of correlations on Rashba SOC-driven transitions in the Kane-Mele model

We investigate, firstly, the competition between the Rashba spin-orbit coupling (SOC) and the intrinsic SOC in Kane-Mele model. For the small intrinsic SOC, we investigate the effects of the Rashba SOC on the touching point of the valence and conduction bands when the ratio of the Rashba SOC to the intrinsic SOC is greater than classical value 2 root 3. For the large intrinsic SOC, we find that the critical ratio of the two SOCs at which the band touching occurs decreases with the increasing intrinsic SOC and the locations of these touching points deviate from pointsKandK ' of the Brillouin zone. Furthermore, effects of the Rashba SOC on these touching points are discussed in detail when the ratio is greater than the critical value. The Rashba SOC-driven topologically trivial and non-trivial transitions are also obtained in the first part of the work. Secondly, using the slave-rotor mean field method we investigate the influences of the correlation on the Rashba SOC-driven topologically trivial and non-trivial transitions in both the charge condensate and Mott regions. The topological Mott insulator with gapped or gapless spin excitations which arises from the interplay of the Rashba SOC and correlations is obtained in the work.