Probability model selection using information-theoretic optimization criterion

Probability models with discrete random variables are often used for probabilistic inference and decision support. A fundamental issue lies in the choice and the validity of the probability model. An information theoretic-based approach fbr probability model selection is discussed. It will be shown that the problem of probability model selection can be formulated as an optimization problem with linear (in)equality constraints and a non-linear objective function. An algorithm for model discovery/selection based on a primal-dual formulation similar to that of the interior point method is presented. The implementation of the algorithm for solving an algebraic system of linear constraints is based on singular value decomposition and the numerical method proposed by Kuenzi, Tzschach and Zehnder. Preliminary comparative evaluation is also discussed.