The combination of tracking and regularization in recursive identification is studied. It is shown that regularization of the information matrix corresponds to a normalization of the covariance matrix, and that several of the proposed methods for dealing with covariance matrix blow up can be interpreted as approximate implementations of covariance matrix normalization.