Distribution-dependent feature selection for deep neural networks

被引:1
作者
Zhao, Xuebin [1 ]
Li, Weifu [1 ]
Chen, Hong [1 ]
Wang, Yingjie [2 ]
Chen, Yanhong [3 ]
John, Vijay [4 ]
机构
[1] Huazhong Agr Univ, Coll Sci, Wuhan 430062, Peoples R China
[2] Huazhong Agr Univ, Coll Informat, Wuhan 430062, Peoples R China
[3] Chinese Acad Sci, Natl Space Sci Ctr, Beijing 100190, Peoples R China
[4] Toyota Technol Inst, Res Ctr Smart Vehicles, Tempaku Ku, 2-12-1 Hisakata, Nagoya, Aichi 4688511, Japan
基金
中国国家自然科学基金;
关键词
Feature selection; Coronal mass ejections; Deep neural networks; Interpretability; Hypothesis-testing; FALSE DISCOVERY RATE; REGRESSION; FILTER;
D O I
10.1007/s10489-021-02663-1
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
While deep neural networks (DNNs) have achieved impressive performance on a wide variety of tasks, the black-box nature hinders their applicability to high-risk, decision-making fields. In such fields, besides accurate prediction, it is also desired to provide interpretable insights into DNNs, e.g., screening important features based on their contributions to predictive accuracy. To improve the interpretability of DNNs, this paper originally proposes a new feature selection algorithm for DNNs by integrating the knockoff technique and the distribution information of irrelevant features. With the help of knockoff features and central limit theorem, we state that the irrelevant feature's statistic follows a known Gaussian distribution under mild conditions. This information is applied in hypothesis testing to discover key features associated with the DNNs. Empirical evaluations on simulated data demonstrate that the proposed method can select more true informative features with higher F-1 scores. Meanwhile, the Friedman test and the post-hoc Nemenyi test are employed to validate the superiority of the proposed method. Then we apply our method to Coronal Mass Ejections (CME) data and uncover the key features which contribute to the DNN-based CME arrival time.
引用
收藏
页码:4432 / 4442
页数:11
相关论文
共 50 条
[41]   Fast, Accurate, and Stable Feature Selection Using Neural Networks [J].
Deraeve, James ;
Alexander, William H. .
NEUROINFORMATICS, 2018, 16 (02) :253-268
[42]   Leveraging Neural Networks and Calibration Measures for Confident Feature Selection [J].
Gharoun, Hassan ;
Yazdanjue, Navid ;
Khorshidi, Mohammad Sadegh ;
Chen, Fang ;
Gandomi, Amir H. .
IEEE TRANSACTIONS ON EMERGING TOPICS IN COMPUTATIONAL INTELLIGENCE, 2025, 9 (03) :2179-2193
[43]   A feature selection approach combining neural networks with genetic algorithms [J].
Huang, Zhi .
AI COMMUNICATIONS, 2019, 32 (5-6) :361-372
[44]   Feature Selection and Outliers Detection with Genetic Algorithms and Neural Networks [J].
Solanas, Agusti ;
Romero, Enrique ;
Gomez, Sergio ;
Sopena, Josep M. ;
Alquezar, Rene ;
Domingo-Ferrer, Josep .
ARTIFICIAL INTELLIGENCE RESEARCH AND DEVELOPMENT, 2005, 131 :41-48
[45]   Convolutional Neural Networks and Feature Selection for BCI with Multiresolution Analysis [J].
Leon, Javier ;
Ortega, Julio ;
Ortiz, Andres .
ADVANCES IN COMPUTATIONAL INTELLIGENCE, IWANN 2019, PT I, 2019, 11506 :883-894
[46]   Integrally Private Model Selection for Deep Neural Networks [J].
Varshney, Ayush K. ;
Torra, Vicenc .
DATABASE AND EXPERT SYSTEMS APPLICATIONS, DEXA 2023, PT II, 2023, 14147 :408-422
[47]   Redundant feature pruning for accelerated inference in deep neural networks [J].
Ayinde, Babajide O. ;
Inanc, Tamer ;
Zurada, Jacek M. .
NEURAL NETWORKS, 2019, 118 :148-158
[48]   Feature Extraction with Deep Neural Networks by a Generalized Discriminant Analysis [J].
Stuhlsatz, Andre ;
Lippel, Jens ;
Zielke, Thomas .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2012, 23 (04) :596-608
[49]   A Multiobjective Sparse Feature Learning Model for Deep Neural Networks [J].
Gong, Maoguo ;
Liu, Jia ;
Li, Hao ;
Cai, Qing ;
Su, Linzhi .
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2015, 26 (12) :3263-3277
[50]   Feature Extraction for Deep Neural Networks Based on Decision Boundaries [J].
Woo, Seongyoun ;
Lee, Chulhee .
PATTERN RECOGNITION AND TRACKING XXVIII, 2017, 10203