P-KDGAN: Progressive Knowledge Distillation with GANs for One-class Novelty Detection

被引:0
|
作者
Zhang, Zhiwei [1 ,2 ,3 ]
Chen, Shifeng [1 ]
Sun, Lei [2 ]
机构
[1] Chinese Acad Sci, Shenzhen Inst Adv Technol, Beijing, Peoples R China
[2] Beijing Inst Technol, Sch Informat & Elect, Beijing, Peoples R China
[3] SIAT, Multimedia Lab, Shenzhen, Guangdong, Peoples R China
来源
PROCEEDINGS OF THE TWENTY-NINTH INTERNATIONAL JOINT CONFERENCE ON ARTIFICIAL INTELLIGENCE | 2020年
基金
中国国家自然科学基金;
关键词
D O I
暂无
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
One-class novelty detection is to identify anomalous instances that do not conform to the expected normal instances. In this paper, the Generative Adversarial Networks (GANs) based on encoderdecoder-encoder pipeline are used for detection and achieve state-of-the-art performance. However, deep neural networks are too over-parameterized to deploy on resource-limited devices. Therefore, Progressive Knowledge Distillation with GANs (PKDGAN) is proposed to learn compact and fast novelty detection networks. The P-KDGAN is a novel attempt to connect two standard GANs by the designed distillation loss for transferring knowledge from the teacher to the student. The progressive learning of knowledge distillation is a two-step approach that continuously improves the performance of the student GAN and achieves better performance than single step methods. In the first step, the student GAN learns the basic knowledge totally from the teacher via guiding of the pretrained teacher GAN with fixed weights. In the second step, joint fine-training is adopted for the knowledgeable teacher and student GANs to further improve the performance and stability. The experimental results on CIFAR-10, MNIST, and FM-NIST show that our method improves the performance of the student GAN by 2.44%, 1.77%, and 1.73% when compressing the computation at ratios of 24.45:1, 311.11:1, and 700:1, respectively.
引用
收藏
页码:3237 / 3243
页数:7
相关论文
共 50 条
  • [21] Combining one-class classifiers for robust novelty detection in gene expression data
    Spinosa, EJ
    de Carvalho, ACPLF
    ADVANCES IN BIOINFORMATICS AND COMPUTATIONAL BIOLOGY, PROCEEDINGS, 2005, 3594 : 54 - 64
  • [22] Automated novelty detection in the WISE survey with one-class support vector machines
    Solarz, A.
    Bilicki, M.
    Gromadzki, M.
    Pollo, A.
    Durkalec, A.
    Wypych, M.
    ASTRONOMY & ASTROPHYSICS, 2017, 606
  • [23] Stabilizing Adversarially Learned One-Class Novelty Detection Using Pseudo Anomalies
    Zaheer, Muhammad Zaigham
    Lee, Jin-Ha
    Mahmood, Arif
    Astrid, Marcella
    Lee, Seung-Ik
    IEEE TRANSACTIONS ON IMAGE PROCESSING, 2022, 31 : 5963 - 5975
  • [24] A modified AdaBoost method for one-class SVM and its application to novelty detection
    Chen, Xue-Fang
    Xing, Hong-Jie
    Wang, Xi-Zhao
    2011 IEEE INTERNATIONAL CONFERENCE ON SYSTEMS, MAN, AND CYBERNETICS (SMC), 2011, : 3506 - 3511
  • [25] Time-series novelty detection using one-class support vector machines
    Ma, JS
    Perkins, S
    PROCEEDINGS OF THE INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS 2003, VOLS 1-4, 2003, : 1741 - 1745
  • [26] Selective Feature Bagging of one-class classifiers for novelty detection in high-dimensional data
    Wang, Biao
    Wang, Wenjing
    Meng, Guanglei
    Meng, Tiankuo
    Song, Bin
    Wang, Yingnan
    Guo, Yuming
    Qiao, Zhihua
    Mao, Zhizhong
    ENGINEERING APPLICATIONS OF ARTIFICIAL INTELLIGENCE, 2023, 120
  • [27] OCmst: One-class novelty detection using convolutional neural network and minimum spanning trees
    La Grassa, Riccardo
    Gallo, Ignazio
    Landro, Nicola
    PATTERN RECOGNITION LETTERS, 2022, 155 : 114 - 120
  • [28] Adaptive Novelty Detection over Contextual Data Streams at the Edge using One-class Classification
    Jodelka, Olga
    Anagnostopoulos, Christos
    Kolomvatsos, Kostas
    2021 12TH INTERNATIONAL CONFERENCE ON INFORMATION AND COMMUNICATION SYSTEMS (ICICS), 2021, : 213 - 219
  • [29] Novelty Detection Methodology based on Multi-Modal One-class Support Vector Machine
    Carino, J. A.
    Zurita, D.
    Picot, A.
    Delgado, M.
    Ortega, J. A.
    Romero-Troncoso, R. J.
    2015 IEEE 10TH INTERNATIONAL SYMPOSIUM ON DIAGNOSTICS FOR ELECTRIC MACHINES, POWER ELECTRONICS AND DRIVES (SDEMPED), 2015, : 184 - 190
  • [30] One-Class Learned Encoder-Decoder Network with Adversarial Context Masking for Novelty Detection
    Jewell, John Taylor
    Khazaie, Vahid Reza
    Mohsenzadeh, Yalda
    2022 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2022), 2022, : 2856 - 2866