How to Fine-Tune BERT for Text Classification?

被引：720

作者：

Sun, Chi ^{[1
]}

Qiu, Xipeng ^{[1
]}

Xu, Yige ^{[1
]}

Huang, Xuanjing ^{[1
]}

机构：

[1] Fudan Univ, Sch Comp Sci, Shanghai Key Lab Intelligent Informat Proc, 825 Zhangheng Rd, Shanghai, Peoples R China

来源：

CHINESE COMPUTATIONAL LINGUISTICS, CCL 2019 | 2019年 / 11856卷

关键词：

Transfer learning; BERT; Text classification;

D O I：

10.1007/978-3-030-32381-3_16

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets.

引用

页码：194 / 206

页数：13

共 50 条

[31] FF-BERT: A BERT-based ensemble for automated classification of web-based text on flash flood events
Wilkho, Rohan Singh
Chang, Shi
Gharaibeh, Nasir G.
ADVANCED ENGINEERING INFORMATICS, 2024, 59
[32] Cross-Domain Text Classification Based on BERT Model
Zhang, Kuan
Hei, Xinhong
Fei, Rong
Guo, Yufan
Jiao, Rui
DATABASE SYSTEMS FOR ADVANCED APPLICATIONS: DASFAA 2021 INTERNATIONAL WORKSHOPS, 2021, 12680 : 197 - 208
[33] Text Classification Research Based on Bert Model and Bayesian Network
Liu, Songsong
Tao, Haijun
Feng, Shiling
2019 CHINESE AUTOMATION CONGRESS (CAC2019), 2019, : 5842 - 5846
[34] Analyzing the Performance of BERT for the Sentiment Classification Task in Bengali Text
Banshal, Sumit Kumar
Uddin, Ashraf
Piryani, Rajesh
ADVANCED NETWORK TECHNOLOGIES AND INTELLIGENT COMPUTING, ANTIC 2023, PT III, 2024, 2092 : 273 - 285
[35] Text Classification by CEFR Levels Using Machine Learning Methods and the BERT Language Model
Lagutina, N. S.
Lagutina, K. V.
Brederman, A. M.
Kasatkina, N. N.
AUTOMATIC CONTROL AND COMPUTER SCIENCES, 2024, 58 (07) : 869 - 878
[36] Text classification problems via BERT embedding method and graph convolutional neural network
Loc Tran
Lam Pham
Tuan Tran
An Mai
2021 INTERNATIONAL CONFERENCE ON ADVANCED TECHNOLOGIES FOR COMMUNICATIONS (ATC 2021), 2021, : 260 - 264
[37] A gating context-aware text classification model with BERT and graph convolutional networks
Gao, Weiqi
Huang, Hao
JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2021, 40 (03) : 4331 - 4343
[38] Emotion Classification of Text Based on BERT and Broad Learning System
Peng, Sancheng
Zeng, Rong
Liu, Hongzhan
Chen, Guanghao
Wu, Ruihuan
Yang, Aimin
Yu, Shui
WEB AND BIG DATA, APWEB-WAIM 2021, PT I, 2021, 12858 : 382 - 396
[39] Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi
Velankar, Abhishek
Patil, Hrushikesh
Joshi, Raviraj
ARTIFICIAL NEURAL NETWORKS IN PATTERN RECOGNITION, ANNPR 2022, 2023, 13739 : 121 - 128
[40] Sensitive Data Detection and Classification in Spanish Clinical Text: Experiments with BERT
Garcia-Pablos, Aitor
Perez, Naiara
Cuadros, Montse
PROCEEDINGS OF THE 12TH INTERNATIONAL CONFERENCE ON LANGUAGE RESOURCES AND EVALUATION (LREC 2020), 2020, : 4486 - 4494

← 1 2 3 4 5 →