Efficient Bayesian Learning of Sparse Deep Artificial Neural Networks

被引:1
作者
Fakhfakh, Mohamed [1 ,2 ]
Bouaziz, Bassem [1 ]
Chaari, Lotfi [2 ]
Gargouri, Faiez [1 ]
机构
[1] Univ Sfax, MIRACL Lab, Sfax, Tunisia
[2] Univ Toulouse, IRIT, INP, Toulouse, France
来源
ADVANCES IN INTELLIGENT DATA ANALYSIS XX, IDA 2022 | 2022年 / 13205卷
关键词
Artificial neural networks; Optimization; Deep learning; LSTM; MCMC; Hamiltonian dynamics;
D O I
10.1007/978-3-031-01333-1_7
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In supervised Machine Learning (ML), Artificial Neural Networks (ANN) are commonly utilized to analyze signals or images for a variety of applications. They are increasingly performing as a strong tool to establish the relationships among data and being successfully applied in science due to their generalization ability, noise and fault tolerance. One of the most difficult aspects of using the learning process is optimization of the network weights. A gradient-based technique with a back-propagation strategy is commonly used for this optimization stage. Regularization is commonly employed for the benefit of efficiency. This optimization gets difficult when non-smooth regularizers are applied, especially to promote sparse networks. Due to differentiability difficulties, traditional gradient-based optimizers cannot be employed. In this paper, we propose an MCMC-based optimization strategy within a Bayesian framework. An effective sampling strategy is designed using Hamiltonian dynamics. The suggested strategy appears to be effective in allowing ANNs with modest complexity levels to achieve high accuracy rates, as seen by promising findings.
引用
收藏
页码:78 / 88
页数:11
相关论文
共 35 条
[1]  
Alpaydin E, 2014, ADAPT COMPUT MACH LE, P1
[2]   Particle Markov chain Monte Carlo methods [J].
Andrieu, Christophe ;
Doucet, Arnaud ;
Holenstein, Roman .
JOURNAL OF THE ROYAL STATISTICAL SOCIETY SERIES B-STATISTICAL METHODOLOGY, 2010, 72 :269-342
[3]  
Angelov Plamen, 2020, MEDRXIV
[4]  
Boudaya Amal, 2020, Impact of Digital Technologies on Public Health in Developed and Developing Countries. 18th International Conference, ICOST 2020. Proceedings. Lecture Notes in Computer Science (LNCS 12157), P69, DOI 10.1007/978-3-030-51517-1_6
[5]   Convolutional Neural Network for Drowsiness Detection Using EEG Signals [J].
Chaabene, Siwar ;
Bouaziz, Bassem ;
Boudaya, Amal ;
Hoekelmann, Anita ;
Ammar, Achraf ;
Chaari, Lotfi .
SENSORS, 2021, 21 (05) :1-19
[6]  
Chaari Lotfi, 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), P1901, DOI 10.1109/ICASSP.2014.6853929
[7]   A Bayesian grouplet transform [J].
Chaari, Lotfi .
SIGNAL IMAGE AND VIDEO PROCESSING, 2019, 13 (05) :871-878
[8]  
Chaari L, 2018, EUR SIGNAL PR CONF, P737, DOI 10.23919/EUSIPCO.2018.8553481
[9]   A Hamiltonian Monte Carlo Method for Non-Smooth Energy Sampling [J].
Chaari, Lotfi ;
Tourneret, Jean-Yves ;
Chaux, Caroline ;
Batatia, Hadj .
IEEE TRANSACTIONS ON SIGNAL PROCESSING, 2016, 64 (21) :5585-5594
[10]   A variational formulation for frame-based inverse problems [J].
Chaux, Caroline ;
Combettes, Patrick L. ;
Pesquet, Jean-Christophe ;
Wajs, Valerie R. .
INVERSE PROBLEMS, 2007, 23 (04) :1495-1518