Knowledge transfer in SVM and neural networks

被引：78

作者：

Vapnik, Vladimir ^{[1
]}

Izmailov, Rauf ^{[2
]}

机构：

[1] Facebook, AI Res Lab, New York, NY USA

[2] Vencore Labs, Basking Ridge, NJ 07920 USA

来源：

ANNALS OF MATHEMATICS AND ARTIFICIAL INTELLIGENCE | 2017年 / 81卷 / 1-2期

关键词：

Intelligent teacher; Privileged information; Similarity control; Knowledge transfer; Knowledge representation; Frames; Support vector machine; Neural network; Classification; Learning theory; Regression;

D O I：

10.1007/s10472-017-9538-x

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

The paper considers general machine learning models, where knowledge transfer is positioned as the main method to improve their convergence properties. Previous research was focused on mechanisms of knowledge transfer in the context of SVM framework; the paper shows that this mechanism is applicable to neural network framework as well. The paper describes several general approaches for knowledge transfer in both SVM and ANN frameworks and illustrates algorithmic implementations and performance of one of these approaches for several synthetic examples.

引用

页码：3 / 19

页数：17

共 50 条

[1] Knowledge transfer in SVM and neural networks
Vladimir Vapnik
Rauf Izmailov
Annals of Mathematics and Artificial Intelligence, 2017, 81 : 3 - 19
[2] Training binary neural networks with knowledge transfer
Leroux, Sam
Vankeirsbilck, Bert
Verbelen, Tim
Simoens, Pieter
Dhoedt, Bart
NEUROCOMPUTING, 2020, 396 : 534 - 541
[3] Knowledge Transfer for Entity Resolution with Siamese Neural Networks
Loster, Michael
Koumarelas, Ioannis
Naumann, Felix
ACM JOURNAL OF DATA AND INFORMATION QUALITY, 2021, 13 (01):
[4] Mixed Transfer Function Neural Networks for Knowledge Acquisition
Khan, M. Imad
Frayman, Yakov
Nahavandi, Saeid
2009 IEEE INTERNATIONAL CONFERENCE ON INDUSTRIAL TECHNOLOGY, VOLS 1-3, 2009, : 1013 - +
[5] TRANSFER KNOWLEDGE FOR HIGH SPARSITY IN DEEP NEURAL NETWORKS
Liu, Wenran
Chen, Xiaogang
Ji, Xiangyang
2017 IEEE GLOBAL CONFERENCE ON SIGNAL AND INFORMATION PROCESSING (GLOBALSIP 2017), 2017, : 1354 - 1358
[6] Knowledge Transfer in Deep Block-Modular Neural Networks
Terekhov, Alexander V.
Montone, Guglielmo
O'Regan, J. Kevin
BIOMIMETIC AND BIOHYBRID SYSTEMS, LIVING MACHINES 2015, 2015, 9222 : 268 - 279
[7] Local Manifold Regularization for Knowledge Transfer in Convolutional Neural Networks
Theodorakopoulos, Ilias
Fotopoulou, Foteini
Economou, George
2020 11TH INTERNATIONAL CONFERENCE ON INFORMATION, INTELLIGENCE, SYSTEMS AND APPLICATIONS (IISA 2020), 2020, : 20 - 27
[8] Knowledge Transfer via Decomposing Essential Information in Convolutional Neural Networks
Lee, Seunghyun
Song, Byung Cheol
IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS, 2022, 33 (01) : 366 - 377
[9] Graph Knowledge Transfer for Offensive Language Identification with Graph Neural Networks
Huang, Yen-Hao
Harryyanto, Kevin
Tsai, Che-Wei
Pornvattanavichai, Ratana
Chen, Yi-Shin
2022 IEEE 23RD INTERNATIONAL CONFERENCE ON INFORMATION REUSE AND INTEGRATION FOR DATA SCIENCE (IRI 2022), 2022, : 216 - 221
[10] Geometric Regularization of Local Activations for Knowledge Transfer in Convolutional Neural Networks
Theodorakopoulos, Ilias
Fotopoulou, Foteini
Economou, George
INFORMATION, 2021, 12 (08)

← 1 2 3 4 5 →