Robustness and performance analysis of subspace-based DOA estimation for rectilinear correlated sources in CES data model

This paper focuses on a theoretical performance analysis of subspace-based algorithms for the localization of spatially correlated rectilinear sources embedded in circular complex elliptically symmetric (C-CES) distributed noise model and also when the observations are non-circular CES (NC-CES) distributed with dependent scatter matrices on the direction of arrival (DOA) parameters. A perturbation analysis has been performed to derive closed-form expressions for the asymptotic covariance matrices of DOA estimates for non-circular subspace-based algorithms in two CES data models. Robustness of subspace-based algorithms is theoretical evaluated using robust covariance matrix estimators (instead of the sample covariance matrix (SCM)). We prove, for the first time, interpretable closed-form expressions of the asymptotic variance of the estimated DOA of two equi-power correlated sources, which allows us to derive a number of properties describing the DOA variance's dependence on signals parameters and non-Gaussian distribution of the noise. Different robustness properties are theoretically analyzed. In particular, we prove in the framework of NC-CES distributed observations, that Tyler's M-estimator enhances the performance for heavy-tailed distributions w.r.t. the SCM, with negligible loss in performance for circular Gaussian distributed observations. Finally, some Monte Carlo illustrations are given for quantifying this robustness and specifying the domain of validity of our theoretical asymptotic results. (C) 2020 Elsevier B.V. All rights reserved.