Global quantitative robustness of regression feed-forward neural networks

被引:0
作者
Werner, Tino [1 ]
机构
[1] Institute for Mathematics, Carl von Ossietzky Universität Oldenburg, Carl-von-Ossietzky-Strasse 9-11, Lower Saxony, Oldenburg
关键词
62F35; 62J02; Breakdown point; Deep learning; Neural networks; Quantitative robustness;
D O I
10.1007/s00521-024-10289-w
中图分类号
学科分类号
摘要
Neural networks are an indispensable model class for many complex learning tasks. Despite the popularity and importance of neural networks and many different established techniques from literature for stabilization and robustification of the training, the classical concepts from robust statistics have rarely been considered so far in the context of neural networks. Therefore, we adapt the notion of the regression breakdown point to regression neural networks and compute the breakdown point for different feed-forward network configurations and contamination settings. In an extensive simulation study, we compare the performance, measured by the out-of-sample loss, by a proxy of the breakdown rate and by the training steps, of non-robust and robust regression feed-forward neural networks in a plethora of different configurations. The results indeed motivate to use robust loss functions for neural network training. © The Author(s) 2024.
引用
收藏
页码:19967 / 19988
页数:21
相关论文
共 63 条
[1]  
Agostinelli C., Leung A., Yohai V.J., Zamar R.H., Robust estimation of multivariate location and scatter in the presence of cellwise and casewise contamination, Test, 24, 3, pp. 441-461, (2015)
[2]  
Alfons A., Croux C., Gelper S., Sparse least trimmed squares regression for analyzing high-dimensional large data sets, Ann Appl Stat, 7, 1, pp. 226-248, (2013)
[3]  
Alqallaf F., Van Aelst S., Yohai V.J., Zamar R.H., Propagation of outliers in multivariate data, Ann Stat, 37, 1, pp. 311-331, (2009)
[4]  
Bartlett P.L., Long P.M., Lugosi G., Tsigler A., Benign overfitting in linear regression, Proc Nat Acad Sci, 117, 48, pp. 30063-30070, (2020)
[5]  
Belagiannis V., Rupprecht C., Carneiro G., Navab N., Robust optimization for deep regression, Proceedings of the IEEE International Conference on Computer Vision, pp. 2830-2838, (2015)
[6]  
Belkin M., Hsu D., Xu J., Two models of double descent for weak features, SIAM J Math Data Sci, 2, 4, pp. 1167-1180, (2020)
[7]  
Berner J., Grohs P., Kutyniok G., Petersen P., The Modern Mathematics of Deep Learning, (2021)
[8]  
Beyer S., Robuste Parameterschätzung für Elektrofahrzeuge, (2019)
[9]  
Biggio B., Nelson B., Laskov P., Poisoning Attacks against Support Vector Machines, (2012)
[10]  
Bloch G., Thomas P., Theilliol D., Accommodation to outliers in identification of non linear SISO systems with neural networks, Neurocomputing, 14, 1, pp. 85-99, (1997)