Hopf bifurcation and Turing instability in a cross-diffusion prey-predator system with group defense behavior

This paper is concerned with a cross-diffusion prey-predator system in which the prey species is equipped with the group defense ability under the Neumann boundary conditions. The tendency of the predator to pursue the prey is expressed in the cross-diffusion coefficient, which can be positive, zero, or negative. We first select the environmental protection of the prey population as a bifurcation parameter. Next, we discuss the Turing instability and the Hopf bifurcation analysis on the proposed cross-diffusion system. We show that the system without cross-diffusion is stable at the constant positive stationary solution but it becomes unstable when the cross-diffusion appears in the system. Furthermore, the stability of bifurcating periodic solutions and the direction of Hopf bifurcation are examined.