Tuning Insulator-Semimetal Transitions in 3D Topological Insulator thin Films by Intersurface Hybridization and In-Plane Magnetic Fields

A pair of Dirac points (analogous to a vortex-antivortex pair) associated with opposite topological numbers (with +/-pi Berry phases) can be merged together through parameter tuning and annihilated to gap the Dirac spectrum, offering a canonical example of a topological phase transition. Here, we report transport studies on thin films of BiSbTeSe2, which is a 3D topological insulator that hosts spin-helical gapless (semimetallic) Dirac fermion surface states for sufficiently thick samples, with an observed resistivity close to h/4e(2) at the charge neutral point. When the sample thickness is reduced to below similar to 10 nm thick, we observe a transition from metallic to insulating behavior, consistent with the expectation that the Dirac cones from the top and bottom surfaces hybridize (analogous to a "merging" in the real space) to give a trivial gapped insulator. Furthermore, we observe that an in-plane magnetic field can drive the system again towards a metallic behavior, with a prominent negative magnetoresistance (up to similar to - 95%) and a temperature-insensitive resistivity close to h/2e(2) at the charge neutral point. The observation is consistent with a predicted effect of an in-plane magnetic field to reduce the hybridization gap (which, if small enough, may be smeared by disorder and give rise to a metallic behavior). A sufficiently strong magnetic field is predicted to restore and split again the Dirac points in the momentum space, inducing a distinct 2D topological semimetal phase with two single-fold Dirac cones of opposite spin-momentum windings.