Automated Multi-Stage Compression of Neural Networks

被引：35

作者：

Gusak, Julia ^{[1
]}

Kholiavchenko, Maksym ^{[2
]}

Ponomarev, Evgeny ^{[1
]}

Markeeva, Larisa ^{[1
]}

Blagoveschensky, Philip ^{[1
,3
]}

Cichocki, Andrzej ^{[1
]}

Oseledets, Ivan ^{[1
]}

机构：

[1] Skolkovo Inst Sci & Technol, Moscow, Russia

[2] Innopolis Univ, Innopolis, Russia

[3] Huawei Noahs Ark Lab, Hong Kong, Peoples R China

来源：

2019 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCVW) | 2019年

基金：

俄罗斯科学基金会;

关键词：

D O I：

10.1109/ICCVW.2019.00306

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Low-rank tensor approximations are very promising for compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.

引用

页码：2501 / 2508

页数：8

共 26 条

[1] Subdominant Dense Clusters Allow for Simple Learning and High Computational Performance in Neural Networks with Discrete Synapses
Baldassi, Carlo
Ingrosso, Alessandro
Lucibello, Carlo
Saglietti, Luca
Zecchina, Riccardo
[J]. PHYSICAL REVIEW LETTERS, 2015, 115 (12)
[2] Denton Emily L, 2014, ADV NEURAL INFORM PR, P1269
[3] Dong X., 2017, MORE IS LESS MORE CO, P5840
[4] The PASCAL Visual Object Classes Challenge: A Retrospective
Everingham, Mark
Eslami, S. M. Ali
Van Gool, Luc
Williams, Christopher K. I.
Winn, John
Zisserman, Andrew
[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION, 2015, 111 (01) : 98 - 136
[5] Gao XY, 2019, CONF REC ASILOMAR C, P930, DOI [10.1109/IEEECONF44664.2019.9048939, 10.1109/ieeeconf44664.2019.9048939]
[6] Harshman R. A., 1970, Foundations of the PARAFAC procedure: Models and conditions for an"explanatory"multimodal factor analysis
[7] He Y., P IEEE INT C COMP VI
[8] He Y., 2018, EUR C COMP VIS SEPT
[9] He Y., P 27 INT JOINT C ART
[10] Most Tensor Problems Are NP-Hard
Hillar, Christopher J.
Lim, Lek-Heng
[J]. JOURNAL OF THE ACM, 2013, 60 (06)

← 1 2 3 →