Rank estimation and redundancy reduction of high-dimensional noisy signals with preservation of rare vectors

In this paper, we address the problem of redundancy-reduction of high-dimensional noisy signals that may contain anomaly (rare) vectors, which we wish to preserve. For example, when applying redundancy reduction techniques to hyperspectral images, it is essential to preserve anomaly pixels for target detection purposes. Since rare-vectors contribute weakly to the l2 -norm of the signal as compared to the noise, l2 -based criteria are unsatisfactory for obtaining a good representation of these vectors. The proposed approach combines l2 and l infinity norms for both signal-subspace and rank determination and considers two aspects: One aspect deals with signal-subspace estimation aiming to minimize the maximum of data-residual l2 -norms, denoted as 2,., for a given rank conjecture. The other determines whether the rank conjecture is valid for the obtained signal-subspace by applying Extreme Value Theory results to model the distribution of the noise l2,infinity -norm. These two operations are performed alternately using a suboptimal greedy algorithm, which makes the proposed approach practically plausible. The algorithm was applied on both synthetically simulated data and on a real hyperspectral image producing better results than common l2 -based methods.