Floor heave is a common disaster in deep soft rock tunnel engineering with high in situ stress. The stress release, transfer and concentration in surrounding rock caused by tunnel excavation are often ignored in the current floor heave mechanism, and only the initial in situ stress state is analyzed. Therefore, in view of the superiority of the combined finite-discrete element method (FDEM) in simulating the elastoplastic continuous deformation, the discontinuous deformation (fracture failure) and the contact action between the rock fragments, FDEM is employed to study evolutionary mechanism of the progressive fracture and swelling deformation of tunnel floor. In addition, the influences of the lateral pressure coefficient, the tensile strength of the surrounding rock mass and the position of the floor on the heave mechanism are also investigated. The simulation results indicate that: (1) The floor heave mechanism is the progressive fracture and swelling deformation of tunnel floor, which can be briefly described as that tunnel excavation leads to a release for the radial stress and a concentration for the tangential stress. When the increased tangential stress exceeds the strength of the rock mass, conjugate shear cracks appear and are accompanied by tensile cracks. The maximum tangential stress continues to evolve into the depth of the intact surrounding rock until it reaches an ultimate equilibrium state with the strength of the rock mass, and the shear cracks also continue to propagate into the deep. The deep fragments push the shallow fragments to move into the tunnel space and create a large number of gaps, resulting in volume expansion and floor heave disaster eventually. (2) According to different lateral pressure coefficients and tensile strengths, five types of floor failure modes can be summarized, but all of them can be considered as the fracture and swelling deformation caused by the maximum tangential concentrated stress. The limitations in the previous floor heave mechanism which did not consider stress evolution phenomenon including stress release, transfer and concentration are improved. A new floor heave mechanism based on progressive fracture and swelling deformation is proposed, which provides a new perspective for the study on the floor heave mechanism.