Dynamics of recurrent modulational instability near its separatrix in nonlinear fiber optics

Separatrix crossing stands for the continuation of the modulation instability into the strong depleted regime and is responsible for the symmetry breaking nature of the recurrence phenomenon. The near-separatrix dynamic for recurrent modulational instability is of great importance because it is closely associated with the formation of rogue breather structures. Here, we have developed the underlying phase-space structure of nonlinear modulation instability by treating the three-mode truncate model as a simple oscillator. This allowed us to reveal the high sensitivity of the switching dynamics characterized by inner or outer trajectories to the initial condition around a separatrix. Our results present a major step forward towards the complete understanding of recurrent modulational instability in a truly conservative setting and can provide a guidance to experimentally measure the involved recursive behaviors.