CancelOut: A Layer for Feature Selection in Deep Neural Networks

被引：30

作者：

Borisov, Vadim ^{[1
]}

Haug, Johannes ^{[1
]}

Kasneci, Gjergji ^{[1
,2
]}

机构：

[1] Eberhard Karls Univ Tubingen, Tubingen, Germany

[2] SCHUFA Holding AG, Wiesbaden, Germany

来源：

ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2019: DEEP LEARNING, PT II | 2019年 / 11728卷

关键词：

Deep learning; Feature ranking; Feature selection; Unsupervised feature selection; Machine learning explainability;

D O I：

10.1007/978-3-030-30484-3_6

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Feature ranking (FR) and feature selection (FS) are crucial steps in data preprocessing; they can be used to avoid the curse of dimensionality problem, reduce training time, and enhance the performance of a machine learning model. In this paper, we propose a new layer for deep neural networks - CancelOut, which can be utilized for FR and FS tasks, for supervised and unsupervised learning. Empirical results show that the proposed method can find feature subsets that are superior to traditional feature analysis techniques. Furthermore, the layer is easy to use and requires adding only a few additional lines of code to a deep learning training loop. We implemented the proposed method using the PyTorch framework and published it online (The code is available at: www.github.com/unnir/CancelOut).

引用

页码：72 / 83

页数：12

共 50 条

[11] Deep feature screening: Feature selection for ultra high-dimensional data via deep neural networks
Li, Kexuan
Wang, Fangfang
Yang, Lingli
Liu, Ruiqi
NEUROCOMPUTING, 2023, 538
[12] Deep Belief Networks with Feature Selection for Sentiment Classification
Ruangkanokmas, Patrawut
Achalakul, Tiranee
Akkarajitsakul, Khajonpong
2016 7TH INTERNATIONAL CONFERENCE ON INTELLIGENT SYSTEMS, MODELLING AND SIMULATION (ISMS), 2016, : 9 - 14
[13] Supervised feature selection through Deep Neural Networks with pairwise connected structure
Huang, Yingkun
Jin, Weidong
Yu, Zhibin
Li, Bing
KNOWLEDGE-BASED SYSTEMS, 2020, 204
[14] A feature selection approach combining neural networks with genetic algorithms
Huang, Zhi
AI COMMUNICATIONS, 2019, 32 (5-6) : 361 - 372
[15] Deep Neural Networks Regularization Using a Combination of Sparsity Inducing Feature Selection Methods
Fatemeh Farokhmanesh
Mohammad Taghi Sadeghi
Neural Processing Letters, 2021, 53 : 701 - 720
[16] Deep Neural Networks Regularization Using a Combination of Sparsity Inducing Feature Selection Methods
Farokhmanesh, Fatemeh
Sadeghi, Mohammad Taghi
NEURAL PROCESSING LETTERS, 2021, 53 (01) : 701 - 720
[17] Selection dynamics for deep neural networks
Liu, Hailiang
Markowich, Peter
JOURNAL OF DIFFERENTIAL EQUATIONS, 2020, 269 (12) : 11540 - 11574
[18] Feature Selection and Extraction for Graph Neural Networks
Acharya, Deepak Bhaskar
Zhang, Huaming
ACMSE 2020: PROCEEDINGS OF THE 2020 ACM SOUTHEAST CONFERENCE, 2020, : 252 - 255
[19] Feature selection using probabilistic neural networks
Hunter, A
NEURAL COMPUTING & APPLICATIONS, 2000, 9 (02) : 124 - 132
[20] Multi-layer Feature Fusion and Selection from Convolutional Neural Networks for Texture Classification
Fradi, Hajer
Fradi, Anis
Dugelay, Jean-Luc
VISAPP: PROCEEDINGS OF THE 16TH INTERNATIONAL JOINT CONFERENCE ON COMPUTER VISION, IMAGING AND COMPUTER GRAPHICS THEORY AND APPLICATIONS - VOL. 4: VISAPP, 2021, : 574 - 581

← 1 2 3 4 5 →