On Relative Entropy Maximum Entropy and design of questionnaires

In this paper we study the means of quantitatively measuring the information brought by the observation during the identification of the laws which govern the random phenomena. From the study on a variable, we build the concept of gain of information on the concept of relative entropy. We show that, at the time of the refinement of the study by disintegration of the states of the character, there is a threshold for the probabilities attached to the various states. This threshold determines the situations where the gain of information is final and those where it is illusory. We show how this study can be extended to the case of several variables. We deduce from it a quantitative method of selection of variables, step by step, respecting the principle of maximum entropy. This method leads to the development, after pre investigation, of parsimonious questionnaires likely to collect the greatest part of information.