An important method for analog-to-digital-converter (ADC) testing is sine wave fitting. In this method, the device is excited with a sine wave, and another sine wave is fitted to the samples at the output of the ADC. The acquisition device can be analyzed by looking at the differences between the fitted signal and the samples. The fit is done using the least-squares (LS) method. If the samples of the error (the difference between the fitted signal and the samples) were random and independent of each other and of the signal, the LS fit would have very good properties. However, when the error is dominated by quantization error, particularly when a low bit number is used or the level of the measured noise is low, these conditions are not fulfilled. The estimation will be biased, and therefore, it must be corrected. The independence of the error samples is more or less true if the sine wave is noisy or dither is used. In these cases, the correction is not necessary. Therefore, it is reasonable to analyze the effect of the potentially unnecessary correction to noisy data, and it is desirable to determine the magnitude of the noise from the measurements. In this paper, these two questions are investigated. The variance of the corrected estimator. is investigated, and a new noise estimation method is developed and analyzed.
