Deep Learning Architecture Improvement Based on Dynamic Pruning and Layer Fusion

被引:3
作者
Li, Qi [1 ,2 ]
Li, Hengyi [1 ,2 ]
Meng, Lin [1 ,2 ]
机构
[1] Ritsumeikan Univ, Dept Elect & Comp Engn, Kusatsu 5258577, Japan
[2] Ritsumeikan Univ, Coll Sci & Engn, 1-1-1 Nojihigashi, Kusatsu 5258577, Japan
关键词
convolutional neural network; architecture improvement; dynamic channel pruning; memory access improvement; NETWORKS;
D O I
10.3390/electronics12051208
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
The heavy workload of current deep learning architectures significantly impedes the application of deep learning, especially on resource-constrained devices. Pruning has provided a promising solution to compressing the bloated deep learning models by removing the redundancies of the networks. However, existing pruning methods mainly focus on compressing the superfluous channels without considering layer-level redundancies, which results in the channel-pruned models still suffering from serious redundancies. To mitigate this problem, we propose an effective compression algorithm for deep learning models that uses both the channel-level and layer-level compression techniques to optimize the enormous deep learning models. In detail, the channels are dynamically pruned first, and then the model is further optimized by fusing the redundant layers. Only a minor performance loss results. The experimental results show that the computations of ResNet-110 are reduced by 80.05%, yet the accuracy is only decreased by 0.72%. Forty-eight convolutional layers could be discarded from ResNet-110 with no loss of performance, which fully demonstrates the efficiency of the proposal.
引用
收藏
页数:14
相关论文
共 46 条
[1]   Combining Weight Pruning and Knowledge Distillation For CNN Compression [J].
Aghli, Nima ;
Ribeiro, Eraldo .
2021 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION WORKSHOPS, CVPRW 2021, 2021, :3185-3192
[2]   Quantized Semantic Segmentation Deep Architecture for Deployment on an Edge Computing Device for Image Segmentation [J].
Ahamad, Afaroj ;
Sun, Chi-Chia ;
Kuo, Wen-Kai .
ELECTRONICS, 2022, 11 (21)
[3]   Deep Learning Techniques for Pattern Recognition in EEG Audio Signal-Processing-Based Eye-Closed and Eye-Open Cases [J].
Almukhtar, Firas Husham ;
Ajwad, Asmaa Abbas ;
Kamil, Amna Shibib ;
Jaleel, Refed Adnan ;
Kamil, Raya Adil ;
Mosa, Sarah Jalal .
ELECTRONICS, 2022, 11 (23)
[4]   Fire Detection Method in Smart City Environments Using a Deep-Learning-Based Approach [J].
Avazov, Kuldoshbay ;
Mukhiddinov, Mukhriddin ;
Makhmudov, Fazliddin ;
Cho, Young Im .
ELECTRONICS, 2022, 11 (01)
[5]   IEEE Robotics and Automation Society Technical Committee on Agricultural Robotics and Automation [J].
Bergerman, Marcel ;
van Henten, Eldert ;
Billingsley, John ;
Reid, John ;
Deng Mingcong .
IEEE ROBOTICS & AUTOMATION MAGAZINE, 2013, 20 (02) :20-+
[6]   Shallowing Deep Networks: Layer-wise Pruning based on Feature Representations [J].
Chen, Shi ;
Zhao, Qi .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2019, 41 (12) :3048-3056
[7]   Robust Pedestrian Detection Based on Multi-Spectral Image Fusion and Convolutional Neural Networks [J].
Chen, Xu ;
Liu, Lei ;
Tan, Xin .
ELECTRONICS, 2022, 11 (01)
[8]   Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks [J].
Chen, Yu-Hsin ;
Emer, Joel ;
Sze, Vivienne .
2016 ACM/IEEE 43RD ANNUAL INTERNATIONAL SYMPOSIUM ON COMPUTER ARCHITECTURE (ISCA), 2016, :367-379
[9]  
cs, CIFAR-10 and CIFAR-100 datasets
[10]   Applications of drone in disaster management: A scoping review [J].
Daud, Sharifah Mastura Syed Mohd ;
Yusof, Mohd Yusmiaidil Putera Mohd ;
Heo, Chong Chin ;
Khoo, Lay See ;
Singh, Mansharan Kaur Chainchel ;
Mahmood, Mohd Shah ;
Nawawi, Hapizah .
SCIENCE & JUSTICE, 2022, 62 (01) :30-42