CancelOut: A Layer for Feature Selection in Deep Neural Networks

被引：30

作者：

Borisov, Vadim ^{[1
]}

Haug, Johannes ^{[1
]}

Kasneci, Gjergji ^{[1
,2
]}

机构：

[1] Eberhard Karls Univ Tubingen, Tubingen, Germany

[2] SCHUFA Holding AG, Wiesbaden, Germany

来源：

ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2019: DEEP LEARNING, PT II | 2019年 / 11728卷

关键词：

Deep learning; Feature ranking; Feature selection; Unsupervised feature selection; Machine learning explainability;

D O I：

10.1007/978-3-030-30484-3_6

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Feature ranking (FR) and feature selection (FS) are crucial steps in data preprocessing; they can be used to avoid the curse of dimensionality problem, reduce training time, and enhance the performance of a machine learning model. In this paper, we propose a new layer for deep neural networks - CancelOut, which can be utilized for FR and FS tasks, for supervised and unsupervised learning. Empirical results show that the proposed method can find feature subsets that are superior to traditional feature analysis techniques. Furthermore, the layer is easy to use and requires adding only a few additional lines of code to a deep learning training loop. We implemented the proposed method using the PyTorch framework and published it online (The code is available at: www.github.com/unnir/CancelOut).

引用

收藏

页码：72 / 83

页数：12

相关论文

共 50 条

[21] Lung and Colon Cancer Classification Using Multiscale Deep Features Integration of Compact Convolutional Neural Networks and Feature Selection [J].

Attallah, Omneya .

TECHNOLOGIES, 2025, 13 (02)

[22] Feature Selection, Deep Neural Network and Trend Prediction [J].

方艳 .

JournalofShanghaiJiaotongUniversity(Science), 2018, 23 (02) :297-307

[23] Feature Selection, Deep Neural Network and Trend Prediction [J].

Fang Y. .

Journal of Shanghai Jiaotong University (Science), 2018, 23 (2) :297-307

[24] Study on feature weight and feature selection in pattern classification neural networks [J].

Li, RP ;

Mukaidono, M ;

Turksen, IB .

INFORMATION INTELLIGENCE AND SYSTEMS, VOLS 1-4, 1996, :1972-1976

[25] An Efficient Image Based Feature Extraction and Feature Selection Model for Medical Data Clustering Using Deep Neural Networks [J].

Ahmed, Mohammed Zaheer ;

Mahesh, Chitraivel .

TRAITEMENT DU SIGNAL, 2021, 38 (04) :1141-1148

[26] Toward the scalability of neural networks through feature selection [J].

Peteiro-Barral, D. ;

Bolon-Canedo, V. ;

Alonso-Betanzos, A. ;

Guijarro-Berdinas, B. ;

Sanchez-Marono, N. .

EXPERT SYSTEMS WITH APPLICATIONS, 2013, 40 (08) :2807-2816

[27] Dynamic Feature Weighting Layer in Artificial Neural Networks [J].

Ozbey, Can ;

Kaplan, Emre ;

Er, Muslum Ersel .

32ND IEEE SIGNAL PROCESSING AND COMMUNICATIONS APPLICATIONS CONFERENCE, SIU 2024, 2024,

[28] IMPROVED MUSIC FEATURE LEARNING WITH DEEP NEURAL NETWORKS [J].

Sigtia, Siddharth ;

Dixon, Simon .

2014 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH AND SIGNAL PROCESSING (ICASSP), 2014,

[29] Disentangling feature and lazy training in deep neural networks [J].

Geiger, Mario ;

Spigler, Stefano ;

Jacot, Arthur ;

Wyart, Matthieu .

JOURNAL OF STATISTICAL MECHANICS-THEORY AND EXPERIMENT, 2020, 2020 (11)

[30] A Kernel Analysis of Feature Learning in Deep Neural Networks [J].

Canatar, Abdulkadir ;

Pehlevan, Cengiz .

2022 58TH ANNUAL ALLERTON CONFERENCE ON COMMUNICATION, CONTROL, AND COMPUTING (ALLERTON), 2022,

← 1 2 3 4 5 →