Semi-supervised deep learning model based on u-wordMixup

被引：0

作者：

Tang H.-L. ^{[1
,3
,4
]}

Song S.-M. ^{[2
]}

Liu X.-Y. ^{[1
]}

Dou Q.-S. ^{[1
,3
,4
]}

Lu M.-Y. ^{[5
]}

机构：

[1] School of Computer Science and Technology, Shandong Technology and Business University, Yantai

[2] School of Information and Electronic Engineering, Shandong Technology and Business University, Yantai

[3] Co-innovation Center of Shandong Colleges and Universities：Future Intelligent Computing, Yantai

[4] Key Laboratory of Intelligent Information Processing in Universities of Shandong, Shandong Technology and Business University, Yantai

[5] Information Science and Technology College, Dalian Maritime University, Dalian

来源：

Kongzhi yu Juece/Control and Decision | 2023年 / 38卷 / 06期

关键词：

data augmentation; deep learning; semi-supervised learning; text categorization;

D O I：

10.13195/j.kzyjc.2021.1789

中图分类号：

学科分类号：

摘要：

When labeled data are deficient, semi-supervised learning uses a large number of unlabeled data to solve the bottleneck problem of labeled data. However, as the unlabeled data and labeled data come from different fields, quality problems of unlabeled data would be callsed, which makes the generalization ability of the model poor and leads to the degradation of classification accuracy. Therefore, based on the wordMixup method, this paper proposes the u-wordMixup method for data augmentation of unlabeled data, and a semi-supervised deep learning model based on the u-wordMixup (SD-uwM) by combining the consistent training framework and the Mean Teacher model. The model utilizes the u-wordMixup method to augment the data of unlabeled data, which can improve the quality of unlabeled data and reduce overfitting under the constraints of supervised cross-entropy and unsupervised consistency loss. The comparative experimental results on the datasets of AGNews, THUCNews and 20 Newsgroups show that the proposed method can improve the generalization ability of the model and also effectively improve the time performance. © 2023 Northeast University. All rights reserved.

引用

页码：1646 / 1652

页数：6

共 17 条

[1] Miyato T, Maeda S I, Koyama M, Et al., Virtual adversarial training: A regularization method for supervised and semi-supervised learning, IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 8, pp. 1979-1993, (2019)
[2] Yang H F, Lin K, Chen C S., Supervised learning of semantics-preserving hash via deep convolutional neural networks, IEEE Transactions on Pattern Analysis and Machine Intelligence, 40, 2, pp. 437-451, (2018)
[3] Tang H L, Dou Q S, Yu L P, Et al., SLDA-TC: A novel text categorization approach based on supervised topic model, Acta Electronica Sinica, 47, 6, pp. 1300-1308, (2019)
[4] Tang H L, Lu M Y, Wu J., Improved AdaBoost algorithm based on vote entropy, Control and Decision, 25, 4, pp. 487-492, (2010)
[5] Zhu J Y, Zhou Z C, Yang H, Et al., Adaptive loss semi-supervised feature selection based on Hessian regularization, Control and Decision, 36, 8, pp. 1862-1870, (2021)
[6] David Berthelot, Nicholas Carlini, Ian Goodfellow, Et al., MixMatch: A holistic approach to Semi-Supervised learning, Advances in Neural Information Processing Systems, 32, pp. 5049-5059, (2019)
[7] Blum A, Mitchell T., Combining labeled and unlabeled data with co-training, Proceedings of the Eleventh Annual Conference on Computational Learning Theory, pp. 92-100, (1998)
[8] Zhou Z H, Li M., Tri-training: Exploiting unlabeled data using three classifiers, IEEE Transactions on Knowledge and Data Engineering, 17, 11, pp. 1529-1541, (2005)
[9] Sajjadi M, Javanmardi M, Tasdizen T., Regularization with stochastic transformations and perturbations for deepsemi-supervised learning, Advances in Neural Information Processing Systems, 29, pp. 1163-1171, (2016)
[10] Clark K, Luong M T, Manning C D, Et al., Semi-supervised sequence modeling with cross-view training[C], Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1914-1925, (2018)

← 1 2 →