CERTIFYING THE ABSENCE OF SPURIOUS LOCAL MINIMA AT INFINITY

被引:1
|
作者
Josz, Cedric [1 ]
Li, Xiaopeng [1 ]
机构
[1] Columbia Univ, Dept Ind Engn & Operat Res, New York, NY 10027 USA
关键词
global optimization; Morse-Sard theorem; subgradient trajectories; VARIATIONAL PRINCIPLE; MATRIX; NETWORKS;
D O I
10.1137/22M1479531
中图分类号
O29 [应用数学];
学科分类号
070104 ;
摘要
When searching for global optima of nonconvex unconstrained optimization problems, it is desirable that every local minimum be a global minimum. This property of having no spurious local minima is true in various problems of interest nowadays, including principal component analysis, matrix sensing, and linear neural networks. However, since these problems are noncoercive, they may yet have spurious local minima at infinity. The classical tools used to analyze the optimization landscape, namely the gradient and the Hessian, are incapable of detecting spurious local minima at infinity. In this paper, we identify conditions that certify the absence of spurious local minima at infinity, one of which is having bounded subgradient trajectories. We check that they hold in several applications of interest.
引用
收藏
页码:1416 / 1439
页数:24
相关论文
共 50 条
  • [1] On the Absence of Spurious Local Minima in Nonlinear Low-Rank Matrix Recovery Problems
    Bi, Yingjie
    Lavaei, Javad
    24TH INTERNATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND STATISTICS (AISTATS), 2021, 130 : 379 - +
  • [2] Structures of Spurious Local Minima in k-Means
    Qian, Wei
    Zhang, Yuqian
    Chen, Yudong
    IEEE TRANSACTIONS ON INFORMATION THEORY, 2022, 68 (01) : 395 - 422
  • [3] Homogeneous polynomials and spurious local minima on the unit sphere
    Jean B. Lasserre
    Optimization Letters, 2022, 16 : 1105 - 1118
  • [4] Spurious Local Minima in Power System State Estimation
    Zhang, Richard Y.
    Lavaei, Javad
    Baldick, Ross
    IEEE TRANSACTIONS ON CONTROL OF NETWORK SYSTEMS, 2019, 6 (03): : 1086 - 1096
  • [5] Homogeneous polynomials and spurious local minima on the unit sphere
    Lasserre, Jean B.
    OPTIMIZATION LETTERS, 2022, 16 (04) : 1105 - 1118
  • [6] Local minima and attractors at infinity for gradient descent learning algorithms
    Blackmore, Kim L.
    Williamson, Robert C.
    Mareels, Iven M.Y.
    Journal of Mathematical Systems, Estimation, and Control, 1996, 6 (02): : 231 - 234
  • [7] Exact guarantees on the absence of spurious local minima for non-negative rank-1 robust principal component analysis
    Fattahi, Salar
    Sojoudi, Somayeh
    Journal of Machine Learning Research, 2020, 21
  • [8] On the Omnipresence of Spurious Local Minima in Certain Neural Network Training Problems
    Christof, Constantin
    Kowalczyk, Julia
    CONSTRUCTIVE APPROXIMATION, 2024, 60 (02) : 197 - 224
  • [9] Exact Guarantees on the Absence of Spurious Local Minima for Non-negative Rank-1 Robust Principal Component Analysis
    Fattahi, Salar
    Sojoudi, Somayeh
    JOURNAL OF MACHINE LEARNING RESEARCH, 2020, 21
  • [10] Spurious Local Minima Are Common for Deep Neural Networks With Piecewise Linear Activations
    Liu, Bo
    IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2024, 35 (04) : 5382 - 5394