Estimation of Mutual Information: A Survey

A common problem found in statistics, signal processing, data analysis and image processing research is the estimation of mutual information, which tends to be difficult. The aim of this survey is threefold: an introduction for those new to the field, an overview for those working in the field and a reference for those searching for literature on different estimation methods. In this paper comparison studies on Mutual information estimation is considered. The paper starts with a description of entropy and mutual information and it closes with a discussion on the performance of different estimation methods and some future challenges.