Improving Convergence Rate of Sign Algorithm using Natural Gradient Method

In lossless audio compression, it is essential for predictive residuals to remain sparse when applying entropy codings. Hence, developing an accurate predictive method is crucial. The sign algorithm (SA) is a conventional method for minimizing the magnitude of residuals; however, it exhibits poor convergence performance compared with the least mean square (LMS) algorithm. To overcome the convergence performance degradation, we proposed novel adaptive algorithms based on a natural gradient: the natural gradient sign algorithm (NGSA) and normalized NGSA (NNGSA). We also propose an efficient update method for the natural gradient based on the AR(p) model. It requires O(p) multiply-add operations at every adaptation step. Through experiments conducted using toy data and real music data, we showed that the proposed algorithms achieve better convergence performance than the SA does. The NNGSA suggested having good compression ability in lossless audio coding.