In the ion beam mixing experiments, eight Fe-Hf-Nb multilayered films, with overall compositions of Fe67Hf22Nb11, Fe67Hf11Nb22, Fe54Hf38Nb8, Fe54Hf30Nb16, Fe54Hf11Nb35, Fe25Hf67Nb8, Fe25Hf50Nb25 and Fe25Hf11Nb64, were irradiated by 200 keV xenon ions to doses ranging from 3x10(14) Xe+/cm(2) to 7x10(15) Xe+/cm(2). The results showed that unique amorphous phases were obtained at designed alloy compositions, falling in the favored glass-forming region deduced from three binary metal sub-systems. Interestingly, at some alloy compositions, the crystal-amorphous-crystal transformations were observed back and forth while varying the irradiation doses. In addition, at the alloy composition of Fe25Hf67Nb8, a metastable FCC phase was formed through an HCP-FCC structural phase transformation and it had a large lattice constant identified to be a=4.51 angstrom. Besides, the formation mechanism of non-equilibrium alloy phases was also discussed in terms of thermodynamics of solids and atomic collision theory.