The calculation of the transmission power spectrum of QSO Ly alpha absorption requires two parameters for the normalization: the continuum F-c and mean transmission (e) over bar (-tau). Traditionally, the continuum is obtained by a polynomial fitting truncating it at a lower order, and the mean transmission is calculated over the entire wavelength range considered. The flux F is then normalized by Fc (e) over tilde (-tau). However, the fluctuations in the transmitted flux are significantly correlated with the local background flux on scales for which the field is intermittent. As a consequence, the normalization of the entire power spectrum by an overall mean transmission (e) over bar (-tau) will overlook the effect of the fluctuation-background correlation upon the powers. In this paper we develop a self-normalization algorithm of the transmission power spectrum based on a multiresolution analysis. This self-normalized power spectrum estimator needs neither a continuum fitting nor a predetermining of the mean transmission. With simulated samples, we show that the self-normalization algorithm can perfectly recover the transmission power spectrum from the flux regardless of how the continuum varies with wavelength. We also show that the self-normalized power spectrum is also properly normalized by the mean transmission. Moreover, this power spectrum estimator is sensitive to the nonlinear behavior of the field. That is, the self-normalized power spectrum estimator can distinguish between fields with or without the fluctuation-background correlation. This cannot be accomplished by the power spectrum with the normalization by an overall mean transmission. Applying this analysis to a real data set of Q1700+642 Ly alpha forest, we demonstrate that the proposed power spectrum estimator can perform correct normalization and effectively reveal the correlation between the fluctuations and background of the transmitted flux on small scales. Therefore, the self-normalized power spectrum would be useful for the discrimination among models without the uncertainties caused by free (or fitting) parameters.