Q-learning based adaptive Kalman filtering for partial model-free dynamic systems

In this article, we propose an adaptive Kalman filtering based on Q-learning for partial model-free dynamic systems. First, a cost function is defined to iteratively update the prior state value when the model parameters are unknown. Then, the observations in a period of time are utilized to improve the accuracy and updating speed of the prior state estimation by means of the multi-innovation least squares. Next, considering that the weight matrix in the cost function will change due to external noise noise and model mismatch, the innovation-based adaptive estimation algorithm is presented to adjust the weight matrix by using the covariance of the information sequence. Finally, the proposed algorithms are applied to estimate the water level of a quadruple water tank system.