A Proximal Algorithm for Optimizing Compositions of Quadratic plus Nonconvex Nonsmooth Functions

Many applications require minimizing the sum of quadratic and nonconvex nonsmooth functions. This problem is commonly referred to the scaled proximal operator. Despite its simple formulation, current approaches suffer from slow convergence or high implementation complexity or both. To overcome these limitations, we develop a new second-order proximal algorithm. Key design involves building and minimizing a series of opportunistically majorized problems along a hybrid Newton direction. The approach computes the Hessian inverse only once, eliminating the need for iteratively updating the approximation as in quasi-Newton methods. Theoretical results on convergence to a critical point and local convergence rate are presented. Numerical results demonstrate that the proposed algorithm not only achieves a faster convergence rate but also tends to converge to a better local optimum.