A block-coordinate approach of multi-level optimization with an application to physics-informed neural networks

被引:1
作者
Gratton, Serge [1 ]
Mercier, Valentin [2 ,3 ]
Riccietti, Elisa [4 ]
Toint, Philippe L. [5 ]
机构
[1] Univ Toulouse, INP ENSEEIHT, IRIT, Toulouse, France
[2] Univ Toulouse, ANITI, CERFACS, IRIT, Toulouse, France
[3] BRL Ingn, Nimes, France
[4] Univ Lyon, ENS Lyon Ecole Normale Super Lyon, INRIA, ENSL,UCBL,CNRS,LIP UMR, Lyon F-5668, France
[5] Univ Namur, Namur Ctr Complex Syst NaXys, Namur, Belgium
关键词
Nonlinear optimization; Multi-level methods; Partial differential equations (PDEs); Physics-informed neural networks (PINNs); Deep learning; TRUST-REGION METHODS; CONVERGENCE; ALGORITHM;
D O I
10.1007/s10589-024-00597-1
中图分类号
C93 [管理学]; O22 [运筹学];
学科分类号
070105 ; 12 ; 1201 ; 1202 ; 120202 ;
摘要
Multi-level methods are widely used for the solution of large-scale problems, because of their computational advantages and exploitation of the complementarity between the involved sub-problems. After a re-interpretation of multi-level methods from a block-coordinate point of view, we propose a multi-level algorithm for the solution of nonlinear optimization problems and analyze its evaluation complexity. We apply it to the solution of partial differential equations using physics-informed neural networks (PINNs) and consider two different types of neural architectures, a generic feedforward network and a frequency-aware network. We show that our approach is particularly effective if coupled with these specialized architectures and that this coupling results in better solutions and significant computational savings.
引用
收藏
页码:385 / 417
页数:33
相关论文
共 48 条
[41]  
Von Planta C., ARXIV
[42]   When and why PINNs fail to train: A neural tangent kernel perspective [J].
Wang, Sifan ;
Yu, Xinling ;
Perdikaris, Paris .
JOURNAL OF COMPUTATIONAL PHYSICS, 2022, 449
[43]   On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks [J].
Wang, Sifan ;
Wang, Hanwen ;
Perdikaris, Paris .
COMPUTER METHODS IN APPLIED MECHANICS AND ENGINEERING, 2021, 384
[44]   The Deep Ritz Method: A Deep Learning-Based Numerical Algorithm for Solving Variational Problems [J].
Weinan, E. ;
Yu, Bing .
COMMUNICATIONS IN MATHEMATICS AND STATISTICS, 2018, 6 (01) :1-12
[45]   Coordinate descent algorithms [J].
Wright, Stephen J. .
MATHEMATICAL PROGRAMMING, 2015, 151 (01) :3-34
[46]   A Multigrid Method for Efficiently Training Video Models [J].
Wu, Chao-Yuan ;
Girshick, Ross ;
He, Kaiming ;
Feichtenhofer, Christoph ;
Krahenbuhl, Philipp .
2020 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR), 2020, :150-159
[47]   Frequency Principle: Fourier Analysis Sheds Light on Deep Neural Networks [J].
Xu, Zhi-Qin John ;
Zhang, Yaoyu ;
Luo, Tao ;
Xiao, Yanyang ;
Ma, Zheng .
COMMUNICATIONS IN COMPUTATIONAL PHYSICS, 2020, 28 (05) :1746-1767
[48]  
Zhang J., 2020, INT C MACHINE LEARNI, P11173