Some notes on mutual information between past and future

We present some new results on the mutual information between past and future for Gaussian stationary sequences. We provide several formulae to calculate this quantity. As a by-product, we establish the so-called reflectrum identity that links partial autocorrelation coefficients and cepstrum coefficients. So as to obtain these results, we provide an account of several regularity conditions for Gaussian stationary processes in terms of properties of the associated Toeplitz and Hankel operators. We discuss conditions under which the mutual information is finite. These results lead us to an interesting perspective towards the definition of long-memory processes. Our result implies that zeros on the unit circle can cause mutual information to be infinite. Examples include fractional autoregressive integrated moving average (ARIMA) models. In addition, we consider a finite sample from a Gaussian stationary sequence. In the expansion of the determinant of its covariance matrix, the Toeplitz matrix, the first and second term are, entropy and mutual information respectively. A form of approximation to the likelihood using entropy and mutual information is presented.