In this article, we address the online state estimation problem of linear discrete-time systems in the presence of inaccurate and slowly time-varying non-Gaussian measurement noise (NGMN). Recently, the variational Bayesian (VB) method has been successfully used to jointly estimate the system state along with the statistics of the unknown Gaussian measurement noise. However, we prove that the original VB method for the non-Gaussian state-space models, modeled by the Gaussian mixture distributions, is analytically intractable. To overcome this problem, we propose a partial VB-based adaptive Gaussian sum Kalman filter, which uses a feedback-based filtering framework to independently calculate the posterior distribution of the state and posterior distribution of the NGMN. Experimental results demonstrate the effectiveness of the proposed filter.