Investigation of possible phase transition of the frustrated spin-1/2 J1-J2-J3 model on the square lattice

The frustrated spin-1/2 J(1)-J(2)-J(3) antiferromagnet with exchange anisotropy on the two-dimensional square lattice is investigated. The exchange anisotropy is presented by eta with 0 = <= eta < 1. The effects of the J(1), J(2), J(3) and anisotropy on the possible phase transition of the Neel state and collinear state are studied comprehensively. Our results indicate that for J(3) > 0 there are upper limits J(3)(c) and eta(c) values. When 0 < J(3) <= J(3)(c) and 0 <= eta <= eta(c), the Neel and collinear states have the same order-disorder transition point at J(2) = J(1)/2. Nevertheless, when the J(3) and eta values beyond the upper limits, it is a paramagnetic phase at J(2) = J(1)/2. For J(3) < 0, in the case of 0 <= eta < 1, the two states always have the same critical temperature as long as J(2) = J(1)/2. Therefore, for J(2) = J(1)/2, under such parameters, a first-order phase transition between the two states for these two cases below the critical temperatures may occur. When J(2) not equal J(1)/2, the Neel and collinear states may also exist, while they have different critical temperatures. When J(2) > J(1)/2, a first-order phase transition between the two states may also occur. However, for J(2) < J(1)/2, the Neel state is always more stable than the collinear state.