The Optimal Regularized Weighted Least-Squares Method for Impulse Response Estimation

The system identification literature has been going through a recent paradigm change with the emergent use of regularization and kernel-based methodologies to identify the process' impulse response. However, the literature is quite scarce when dealing with processes that possess colored additive output noise. In this case, the current alternative is to identify a system predictor instead, which can be somewhat unfavorable in situations where the process' model is strictly necessary. So, the main objective of this paper is to introduce a novel regularized system identification methodology that has been specifically developed for the colored output noise scenario. Such methodology is based on the Bayesian perspective of the identification procedure, and it results in the regularized weighted least-squares method, which can be interpreted as an extension of the well-known regularized least squares. The paper also presents the method's statistical properties, optimal choices, and parametrization structures for both the regularization and weighting matrices, along with a dedicated algorithm to estimate these matrices. Finally, Monte Carlo simulations are performed to demonstrate the method's efficiency.