Scalable and globally convergent algorithm for sufficient dimension reduction

Sufficient Dimension Reduction (SDR) is a powerful approach for analyzing high-dimensional data, where the goal to represent covariates by a minimal set of their linear projections that still capture the necessary information for regression analysis of the response. However, many existing SDR methods employ a generalized eigen decomposition of method-specific kernel matrix, which is a non-convex optimization problem and requires significant computation involving large matrix products and decomposition. In this paper, we propose an iterative least squares formulation for SDR, which solves each least squares problem approximately. We combine this formulation with state-of-the-art stochastic gradient descent methods to propose a scalable and globally convergent algorithm for SDR. To the best of our knowledge, this is the first stochastic algorithm proposed for SDR. Through extensive simulations, we demonstrate that our method achieves competitive empirical performance.