Pruning graph neural networks by evaluating edge properties

被引:8
作者
Wang, Li [1 ,2 ]
Huang, Wei [2 ]
Zhang, Miao [3 ]
Pan, Shirui [4 ]
Chang, Xiaojun [2 ]
Su, Steven Weidong [1 ,2 ]
机构
[1] Shandong First Med Univ & Shandong Acad Med Sci, Coll Artif Intelligence & Big Data Med Sci, 6699 Qingdao Rd, Jinan 250000, Shandong, Peoples R China
[2] Univ Technol Sydney, Fac Engn & IT, 81 Broadway, Ultimo, NSW 2007, Australia
[3] Aalborg Univ, Comp Sci Dept, Fredrik Bajers Vej 7K, DK-9220 Aalborg, Denmark
[4] Griffith Univ, Sch Informat & Commun Technol, Parklands Dr, Southport, Qld 4222, Australia
关键词
Graph neural networks; Network pruning; Model compression;
D O I
10.1016/j.knosys.2022.109847
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
The emergence of larger and deeper graph neural networks (GNNs) makes their training and inference increasingly expensive. Existing GNN pruning methods simultaneously prune the graph adjacency matrix and the model weights on a pretrained neural network by directly leveraging the lottery-ticket hypothesis, but the benefits of such methods are mainly via weight pruning, and methods based on saliency metrics struggle to outperform random pruning when pruning only the graph adjacency matrix. This motivates us to use different scoring standards for graph edges and network weights during GNN pruning. Thus, rather than measuring the importance of graph edges based on saliency metrics, we formulate the performance of GNNs mathematically with respect to the properties of their edges, elucidating how the performance drop can be avoided by pruning negative edges and nonbridges. This leads to our simple but effective two-step method for GNN pruning, leveraging the saliency metrics for the network pruning while sparsifying the graph with preservation of the loss performance. Experimental results show the effectiveness and efficiency of the proposed method on both small-scale graph datasets (Cora, Citeseer, and PubMed) and a large-scale dataset (Ogbn-ArXiv), where our method saves up to 98% of floating-point operations per second (FLOPs) on the small graphs and 94% of FLOPs on the large one, with no significant drop in accuracy. (c) 2022 Elsevier B.V. All rights reserved.
引用
收藏
页数:10
相关论文
共 50 条
[21]   Heterogeneous Molecular Graph Neural Networks for Predicting Molecule Properties [J].
Shui, Zeren ;
Karypis, George .
20TH IEEE INTERNATIONAL CONFERENCE ON DATA MINING (ICDM 2020), 2020, :492-500
[22]   AnisoGNN: Graph neural networks generalizing to anisotropic properties of polycrystals [J].
Hu, Guangyu ;
Latypov, Marat, I .
COMPUTATIONAL MATERIALS SCIENCE, 2024, 243
[23]   Neural Pooling for Graph Neural Networks [J].
Harsha, Sai Sree ;
Mishra, Deepak .
PATTERN RECOGNITION AND MACHINE INTELLIGENCE, PREMI 2021, 2024, 13102 :171-180
[24]   Iterative clustering pruning for convolutional neural networks [J].
Chang, Jingfei ;
Lu, Yang ;
Xue, Ping ;
Xu, Yiqun ;
Wei, Zhen .
KNOWLEDGE-BASED SYSTEMS, 2023, 265
[25]   Activation-Based Pruning of Neural Networks [J].
Ganguli, Tushar ;
Chong, Edwin K. P. ;
Werner, Frank .
ALGORITHMS, 2024, 17 (01)
[26]   Sparse optimization guided pruning for neural networks [J].
Shi, Yong ;
Tang, Anda ;
Niu, Lingfeng ;
Zhou, Ruizhi .
NEUROCOMPUTING, 2024, 574
[27]   Structured pruning of neural networks for constraints learning [J].
Cacciola, Matteo ;
Frangioni, Antonio ;
Lodi, Andrea .
OPERATIONS RESEARCH LETTERS, 2024, 57
[28]   An iterative pruning algorithm for feedforward neural networks [J].
Castellano, G ;
Fanelli, AM ;
Pelillo, M .
IEEE TRANSACTIONS ON NEURAL NETWORKS, 1997, 8 (03) :519-531
[29]   Fast Convex Pruning of Deep Neural Networks [J].
Aghasi, Alireza ;
Abdi, Afshin ;
Romberg, Justin .
SIAM JOURNAL ON MATHEMATICS OF DATA SCIENCE, 2020, 2 (01) :158-188
[30]   Blending Pruning Criteria for Convolutional Neural Networks [J].
He, Wei ;
Huang, Zhongzhan ;
Liang, Mingfu ;
Liang, Senwei ;
Yang, Haizhao .
ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2021, PT IV, 2021, 12894 :3-15