Graphical Granger Causality by Information-Theoretic Criteria

Causal inference by a graphical Granger model (GGM) among p variables is typically solved by p penalized linear regression problems in time series with a given lag. In practice however, the estimates of a penalized linear regression after a finite number of steps can be still far from the optimum. Furthermore, the selection of the regularization parameter, influencing the precision of the model is not trivial, especially when the corresponding design matrix is super-collinear. In this paper, for the first time we concept a graphical Granger model as an instance of combinatorial optimization. Computing maximum likelihood (ML) estimates of the regression coefficients and of the variance for each of p variables we propose an information-theoretic graphical Granger model (ITGGM). In the sense of information theory, the criterion to be minimized is the complexity of the class of the selected models together with the complexity of the data set. Following this idea, we propose four various information-theoretic (IT) objective functions based on stochastic complexity, on minimum message length, on Akaike and on Bayesian information criterion. To find their minima we propose a genetic algorithm operating with populations of subsets of regressor variables. The feature selection by the ITGGM with any of the functions is parameter-free in the sense that beside the ML estimates which are for each and within the model constant, no adjustable parameter is added into these objective functions. We further provide a theoretical analysis of the convergence properties of the GGM with the proposed IT functions. We test the performance of the functions in terms of F-1 measure with respect to two common penalized GGMs on synthetic and real data. The experiments demonstrate the significant superiority of the IT criteria in terms of F-1 measure over the two alternatives of the penalized GGM for Granger causal inference.