Compressing neural networks with two-layer decoupling

被引：0

作者：

De Jonghe, Joppe ^{[1
]}

Usevich, Konstantin ^{[2
]}

Dreesen, Philippe ^{[3
]}

Ishteva, Mariya ^{[1
]}

机构：

[1] Katholieke Univ Leuven, Dept Comp Sci, Geel, Belgium

[2] Univ Lorraine, CNRS, Nancy, France

[3] Maastricht Univ, DACS, Maastricht, Netherlands

来源：

2023 IEEE 9TH INTERNATIONAL WORKSHOP ON COMPUTATIONAL ADVANCES IN MULTI-SENSOR ADAPTIVE PROCESSING, CAMSAP | 2023年

关键词：

tensor; tensor decomposition; decoupling; compression; neural network; MODEL COMPRESSION; ACCELERATION;

D O I：

10.1109/CAMSAP58249.2023.10403509

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

The single-layer decoupling problem has recently been used for the compression of neural networks. However, methods that are based on the single-layer decoupling problem only allow the compression of a neural network to a single flexible layer. As a result, compressing more complex networks leads to worse approximations of the original network due to only having one flexible layer. Having the ability to compress to more than one flexible layer thus allows to better approximate the underlying network compared to compression into only a single flexible layer. Performing compression into more than one flexible layer corresponds to solving a multi-layer decoupling problem. As a first step towards general multi-layer decoupling, this work introduces a method for solving the two-layer decoupling problem in the approximate case. This method enables the compression of neural networks into two flexible layers.

引用

页码：226 / 230

页数：5

共 50 条

[31] Two-layer Residual Feature Fusion for Object Detection
Choi, Jaeseok
Lee, Kyoungmin
Jeong, Jisoo
Kwak, Nojun
ICPRAM: PROCEEDINGS OF THE 8TH INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION APPLICATIONS AND METHODS, 2019, : 352 - 359
[32] EvoPrunerPool: An Evolutionary Pruner using Pruner Pool for Compressing Convolutional Neural Networks
Subramanian, S. Sujit
Arvindram, K.
Velayutham, C. Shunmuga
Sathya, Madhusoodhan
Sengodan, Nathiya
Kosuri, Divesh
Satvik, Arvapalli Sai
Thangavelu, S.
Jeyakumar, G.
PROCEEDINGS OF THE 2023 GENETIC AND EVOLUTIONARY COMPUTATION CONFERENCE COMPANION, GECCO 2023 COMPANION, 2023, : 2136 - 2143
[33] Direct adaptive decoupling control of nonlinear systems based on neural networks and multiple models
Fu, Yue
Chai, Tianyou
WCICA 2006: SIXTH WORLD CONGRESS ON INTELLIGENT CONTROL AND AUTOMATION, VOLS 1-12, CONFERENCE PROCEEDINGS, 2006, : 2475 - 2479
[34] A Hybrid Model Based on a Two-Layer Decomposition Approach and an Optimized Neural Network for Chaotic Time Series Prediction
Xu, Xinghan
Ren, Weijie
SYMMETRY-BASEL, 2019, 11 (05):
[35] Optical hardware implementation of the two-layer neural network with the pre-processing unit for invariant pattern recognition
Evtikhiev, NN
Onyky, BN
Repin, DV
Scherbakov, IB
Starikov, RS
Zabulonov, MI
OPTICAL PATTERN RECOGNITION VII, 1996, 2752 : 281 - 289
[36] Neural self-compressor: Collective interpretation by compressing multi-layered neural networks into non-layered networks
Kamimura, Ryotaro
NEUROCOMPUTING, 2019, 323 : 12 - 36
[37] Recurrent neural networks in decoupling control of multivariable nonlinear systems
Nikolaou, M
Hanagandi, V
CHEMICAL ENGINEERING COMMUNICATIONS, 1995, 136 : 201 - 216
[38] A constructive algorithm to solve "convex recursive deletion" (CoRD) classification problems via two-layer perceptron networks
Cabrelli, C
Molter, U
Shonkwiler, R
IEEE TRANSACTIONS ON NEURAL NETWORKS, 2000, 11 (03): : 811 - 816
[39] Compressing Low Precision Deep Neural Networks Using Sparsity-Induced Regularization in Ternary Networks
Faraone, Julian
Fraser, Nicholas
Gambardella, Giulio
Blott, Michaela
Leong, Philip H. W.
NEURAL INFORMATION PROCESSING (ICONIP 2017), PT II, 2017, 10635 : 393 - 404
[40] Filter pruning with a feature map entropy importance criterion for convolution neural networks compressing
Wang, Jielei
Jiang, Ting
Cui, Zongyong
Cao, Zongjie
NEUROCOMPUTING, 2021, 461 : 41 - 54

← 1 2 3 4 5 →