Model selection for wavelet-based signal estimation

There has been a growing interest in wavelet-based methods for signal estimation from noisy samples. Signal denoising involves calculating discrete wavelet transform (using training samples) and then discarding 'insignificant' wavelet coefficients (presumably corresponding to noise). Various wavelet thresholding heuristics for discarding insignificant wavelets have been recently proposed [Bruce et al, 1996; Donoho, 1993; Donoho and Johnstone, 1994]. These methods are conceptually based on asymptotic results for linear models, but also take into account special properties of wavelet basis functions. Wavelet thresholding represents a special case of model selection; hence we compare popular wavelet thresholding methods with model selection using VC generalization bounds developed for finite samples [Vapnik, 1982]. Since wavelet methods are linear (in parameters), VC bounds can be rigorously applied, i.e. the VC-dimension of linear models can be accurately estimated. Successful application of VC-theory to wavelet denoising also requires specification of a suitable structure on a set of wavelet basis functions. We propose such a structure suitable for orthogonal basis functions, which includes wavelets as a special case. The combination of the proposed structure with VC bounds yields a new powerful method for signal estimation with wavelets. Our comparisons indicate that using VC bounds for model selection gives uniformly better results than other wavelet thresholding methods under small sample! high noise setting. On the other hand, with large samples model selection becomes trivial, and most reasonable methods (including wavelet thresholding heuristics) perform reasonably well.