A Continuous Exact l0 Penalty (CEL0) for Least Squares Regularized Problem

Within the framework of the l(0) regularized least squares problem, we focus, in this paper, on nonconvex continuous penalties approximating the l(0)-norm. Such penalties are known to better promote sparsity than the l(1) convex relaxation. Based on some results in one dimension and in the case of orthogonal matrices, we propose the continuous exact l(0) penalty (CEL0) leading to a tight continuous relaxation of the l(2) - l(0) problem. The global minimizers of the CEL0 functional contain the global minimizers of l(2) - l(0), and from each global minimizer of CEL0 one can easily identify a global minimizer of l(2) - l(0). We also demonstrate that from each local minimizer of the CEL0 functional, a local minimizer of l(2) - l(0) is easy to obtain. Moreover, some strict local minimizers of the initial functional are eliminated with the proposed tight relaxation. Then solving the initial l(2) - l(0) problem is equivalent, in a sense, to solving it by replacing the l(0)-norm with the CEL0 which provides better properties for the objective function in terms of minimization, such as the continuity and the convexity with respect to each direction of the standard RN basis, although the problem remains nonconvex. Finally, recent nonsmooth nonconvex algorithms are used to address this relaxed problem within a macro algorithm ensuring the convergence to a critical point of the relaxed functional which is also a (local) optimum of the initial problem.