Sequential Independent Component Analysis Density Estimation

A problem of multivariate probability density function estimation by exploiting linear independent components analysis (ICA) is addressed. Historically, ICA density estimation was initially proposed under the name projection pursuit density estimation (PPDE) and two basic methods, named forward and backward, were published. We derive a modification of the forward PPDE method, which avoids a computationally demanding optimization involving Monte Carlo sampling of the original method. The results of the experiments show that the proposed method presents an attractive choice for density estimation, which is pronounced for a small number of training observations. Under such conditions, our method usually outperforms model-based Gaussian mixture model. We also found that our method obtained better results than the backward PPDE methods in the situation of nonfactorizable underlying density functions. The proposed method has demonstrated a competitive performance compared with the support vector machine and the extreme learning machine in some real classification tasks.