Linear Programming-Based Sparse Kernel Regression with L1-Norm Minimization for Nonlinear System Modeling

被引:0
作者
Liu, Xiaoyong [1 ]
Yan, Genglong [1 ]
Zhang, Fabin [1 ]
Zeng, Chengbin [1 ]
Tian, Peng [1 ]
机构
[1] Moutai Inst, Automat Dept Brewing Engn, Renhuai 564507, Peoples R China
基金
中国国家自然科学基金;
关键词
linear programming; structural risk minimization; L1-norm on approximation error; sparse kernel regression; nonlinear system modeling; IDENTIFICATION;
D O I
10.3390/pr12112358
中图分类号
TQ [化学工业];
学科分类号
0817 ;
摘要
This paper integrates L1-norm structural risk minimization with L1-norm approximation error to develop a new optimization framework for solving the parameters of sparse kernel regression models, addressing the challenges posed by complex model structures, over-fitting, and limited modeling accuracy in traditional nonlinear system modeling. The first L1-norm regulates the complexity of the model structure to maintain its sparsity, while another L1-norm is essential for ensuring modeling accuracy. In the optimization of support vector regression (SVR), the L2-norm structural risk is converted to an L1-norm framework through the condition of non-negative Lagrange multipliers. Furthermore, L1-norm optimization for modeling accuracy is attained by minimizing the maximum approximation error. The integrated L1-norm of structural risk and approximation errors creates a new, simplified optimization problem that is solved using linear programming (LP) instead of the more complex quadratic programming (QP). The proposed sparse kernel regression model has the following notable features: (1) it is solved through relatively simple LP; (2) it effectively balances the trade-off between model complexity and modeling accuracy; and (3) the solution is globally optimal rather than just locally optimal. In our three experiments, the sparsity metrics of SVs% were 2.67%, 1.40%, and 0.8%, with test RMSE values of 0.0667, 0.0701, 0.0614 (sinusoidal signal), and 0.0431 (step signal), respectively. This demonstrates the balance between sparsity and modeling accuracy.
引用
收藏
页数:17
相关论文
共 24 条
  • [1] Robust solutions of Linear Programming problems contaminated with uncertain data
    Ben-Tal, A
    Nemirovski, A
    [J]. MATHEMATICAL PROGRAMMING, 2000, 88 (03) : 411 - 424
  • [2] Subspace identification of Hammerstein systems using least squares support vector machines
    Goethals, I
    Pelckmans, K
    Suykens, JAK
    De Moor, B
    [J]. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, 2005, 50 (10) : 1509 - 1519
  • [3] Nonlinear system modeling using a self-organizing recurrent radial basis function neural network
    Han, Hong-Gui
    Guo, Ya-Nan
    Qiao, Jun-Fei
    [J]. APPLIED SOFT COMPUTING, 2018, 71 : 1105 - 1116
  • [4] Recurrent fuzzy neural network backstepping control for the prescribed output tracking performance of nonlinear dynamic systems
    Han, Seong-Ik
    Lee, Jang-Myung
    [J]. ISA TRANSACTIONS, 2014, 53 (01) : 33 - 43
  • [5] A novel unscented Kalman filter for recursive state-input-system identification of nonlinear systems
    Lei, Ying
    Xia, Dandan
    Erazo, Kalil
    Nagarajaiah, Satish
    [J]. MECHANICAL SYSTEMS AND SIGNAL PROCESSING, 2019, 127 : 120 - 135
  • [6] An active sparse polynomial chaos expansion approach based on sequential relevance vector machine
    Li, Yangtian
    Luo, Yangjun
    Zhong, Zheng
    [J]. COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING, 2024, 418
  • [7] Liu X., 2020, Sci. Technol. Eng, V20, P7804
  • [8] Structural learning in artificial neural networks using sparse optimization
    Manngard, Mikael
    Kronqvist, Jan
    Boling, Jari M.
    [J]. NEUROCOMPUTING, 2018, 272 : 660 - 667
  • [9] A self-organizing fuzzy neural network with hybrid learning algorithm for nonlinear system modeling
    Meng, Xi
    Zhang, Yin
    Quan, Limin
    Qiao, Junfei
    [J]. INFORMATION SCIENCES, 2023, 642
  • [10] A novel thermo-mechanical local damage model for quasi-brittle fracture analysis
    Pham, Manh Van
    Nguyen, Minh Ngoc
    Bui, Tinh Quoc
    [J]. THEORETICAL AND APPLIED FRACTURE MECHANICS, 2024, 130