Direct Importance Estimation with Gaussian Mixture Models

The ratio of two probability densities is called the importance and its estimation has gathered a great deal of attention these days since the importance can be used for various data processing purposes. In this paper, we propose a new importance estimation method using Gaussian mixture models (GMMs). Our method is an extention of the Kullback-Leibler importance estimation procedure (KLIEP), an importance estimation method using linear or kernel models. An advantage of GMMs is that covariance matrices can also be learned through an expectation-maximization procedure, so the proposed method-which we call the Gaussian mixture KLIEP (GM-KLIEP)-is expected to work well when the true importance function has high correlation. Through experiments, we show the validity of the proposed approach.