The installation error of a wave generator is a common source of error in deformation measurement of a flexspline. This installation error leads to a large deviation in the functional relationship between the deformation of the flexspline and rotation angle. In this study, we attempt to solve the problem of the installation error between the center of the wave generator and center of rotation by analyzing the measurement error of the flexspline and proposing a compensation method for the radial deformation of the flexspline. First, based on the principle of coordinate change, a mathematical model of eccentricity error characterization is established to obtain the installation eccentricity and actual structural parameters of the wave generator. Second, based on the eccentric radial deformation function of the wave generator, a correction model for the radial deformation error of the flexspline is developed to correct and compensate for the actual radial deformation function of the flexspline under eccentric installation conditions. The experimental results indicate that the peak-to-valley deviation of the flexspline deformation function corresponds to 0.134 mm under the effect of installation error, which significantly differs from the theoretical value. Following correction and compensation using the method proposed in the study, the error obtained is approximately 0.012 mm, and the radial deformation of the flexspline differs from the theoretical value. The trend is essentially identical. The method can effectively improve measurement accuracy of the radial deformation of the flexspline and establish an experimental and theoretical foundation to optimize tooth profile parameters of the harmonic reducer. © 2021, Science Press. All right reserved.