Learning Bayesian networks with an approximated MDL score

In this paper we present an approximation to the mutual information between a single variable and a set of variables. The main aim of our approach is to reduce the amount of sufficient statistics (i.e. frequency counts) required to calculate the mutual information. To do so, we use the chain rule and assume different independence statements between variables. We will use our approximation to calculate the MDL of a given Bayesian network. We will show that our approximated approach to the MDL measure is score equivalent and we will use it in order to learn Bayesian networks from data. We will experimentally see that learning algorithms that use our approach obtain high quality Bayesian networks. We also note that our approach can be used in any information based measures.