A Novel Two-stage Learning Pipeline for Deep Neural Networks

被引:3
|
作者
Ding, Chunhui [1 ]
Hu, Zhengwei [1 ]
Karmoshi, Saleem [1 ]
Zhu, Ming [1 ]
机构
[1] Univ Sci & Technol China, Hefei, Anhui, Peoples R China
关键词
DNN; Two-stage structure; Big data;
D O I
10.1007/s11063-017-9578-6
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
In this work, a training method was proposed for Deep Neural Networks (DNNs) based on a two-stage structure. Local DNN models are trained in all local machines and uploaded to the center with partial training data. These local models are integrated as a new DNN model (combination DNN). With another DNN model (optimization DNN) connected, the combination DNN forms a global DNN model in the center. This results in greater accuracy than local DNN models with smaller amounts of data uploaded. In this case, the bandwidth of the uploaded data is saved, and the accuracy is maintained as well. Experiments are conducted on MNIST dataset, CIFAR-10 dataset and LFW dataset. The results show that with less training data uploaded, the global model produces greater accuracy than local models. Specifically, this method focuses on condition of big data.
引用
收藏
页码:159 / 169
页数:11
相关论文
共 50 条
  • [31] Deep Model Compression via Two-Stage Deep Reinforcement Learning
    Zhan, Huixin
    Lin, Wei-Ming
    Cao, Yongcan
    MACHINE LEARNING AND KNOWLEDGE DISCOVERY IN DATABASES, 2021, 12975 : 238 - 254
  • [32] Two-Stage Training of Graph Neural Networks for Graph Classification
    Do, Manh Tuan
    Park, Noseong
    Shin, Kijung
    NEURAL PROCESSING LETTERS, 2023, 55 (03) : 2799 - 2823
  • [33] Two-stage data encryption using chaotic neural networks
    Srivastava, Gautam
    Kumar, C. N. S. Vinoth
    Kavitha, V
    Parthiban, N.
    Venkataraman, Revathi
    JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2020, 38 (03) : 2561 - 2568
  • [34] Two-stage pyramidal convolutional neural networks for image colorization
    Wei, Yu-Jen
    Wei, Tsu-Tsai
    Kuo, Tien-Ying
    Su, Po-Chyi
    APSIPA TRANSACTIONS ON SIGNAL AND INFORMATION PROCESSING, 2021, 10 (10)
  • [35] Two-Stage Training of Graph Neural Networks for Graph Classification
    Manh Tuan Do
    Noseong Park
    Kijung Shin
    Neural Processing Letters, 2023, 55 : 2799 - 2823
  • [36] A Two-Stage Convolutional Neural Networks for Lung Nodule Detection
    Cao, Haichao
    Liu, Hong
    Song, Enmin
    Ma, Guangzhi
    Xu, Xiangyang
    Jin, Renchao
    Liu, Tengying
    Hung, Chih-Cheng
    IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, 2020, 24 (07) : 2006 - 2015
  • [37] A Novel Two-Stage Impulse Noise Removal Technique Based on Neural Networks and Fuzzy Decision
    Liang, Sheng-Fu
    Lu, Shih-Mao
    Chang, Jyh-Yeong
    Lin, Chin-Teng
    IEEE TRANSACTIONS ON FUZZY SYSTEMS, 2008, 16 (04) : 863 - 873
  • [38] Automatic lung segmentation in dynamic thoracic MRI using two-stage deep convolutional neural networks
    Xie, Lipeng
    Udupa, Jayaram K.
    Tong, Yubing
    McDonough, Joseph M.
    Wu, Caiyun
    Lott, Carina
    Anari, Jason B.
    Cahill, Patrick J.
    Torigian, Drew A.
    MEDICAL IMAGING 2022: IMAGE PROCESSING, 2022, 12032
  • [39] Signal Anomaly Detection of Bridge SHM System Based on Two-Stage Deep Convolutional Neural Networks
    Li, Sheng
    Jin, Liang
    Qiu, Yang
    Zhang, Mimi
    Wang, Jie
    STRUCTURAL ENGINEERING INTERNATIONAL, 2023, 33 (01) : 74 - 83
  • [40] Two-Stage Deep Learning Segmentation for Tiny Brain Regions
    Ren, Yan
    Zheng, Xiawu
    Ji, Rongrong
    Chen, Jie
    PATTERN RECOGNITION AND COMPUTER VISION, PRCV 2023, PT XIII, 2024, 14437 : 174 - 184