To manage the complexities of vibration reasons, a new method to predict the vibration and analyze the reliability of the turbine rotor is proposed in this paper. Based on analyzing the vibration reasons, the measuring positions of vibration are obtained, and then the rotor will be periodic measured under the normal operation condition to get the test date, namely the amplitude of vibration. Based on the amplitude, the grey model optimized by BP neural network is established. Finally, a case study has been conducted, which proves that the model is valid and applicable; especially it could find vibration fault earlier in the operation of the rotor and determine the maintenance program which can ensure the security reliability of the turbines.