How to Fine-Tune BERT for Text Classification?

被引：720

作者：

Sun, Chi ^{[1
]}

Qiu, Xipeng ^{[1
]}

Xu, Yige ^{[1
]}

Huang, Xuanjing ^{[1
]}

机构：

[1] Fudan Univ, Sch Comp Sci, Shanghai Key Lab Intelligent Informat Proc, 825 Zhangheng Rd, Shanghai, Peoples R China

来源：

CHINESE COMPUTATIONAL LINGUISTICS, CCL 2019 | 2019年 / 11856卷

关键词：

Transfer learning; BERT; Text classification;

D O I：

10.1007/978-3-030-32381-3_16

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets.

引用

页码：194 / 206

页数：13

共 50 条

[21] Analyzing Fine-Tune Pre-trained Models for Detecting Cucumber Plant Growth
Hari, Pragya
Singh, Maheshwari Prasad
ADVANCED NETWORK TECHNOLOGIES AND INTELLIGENT COMPUTING, ANTIC 2022, PT II, 2023, 1798 : 510 - 521
[22] Fault Text Classification of Rotating Machine Based BERT
Chen Ling
Liu Yimin
Ji Lianlian
PROCEEDINGS OF THE 33RD CHINESE CONTROL AND DECISION CONFERENCE (CCDC 2021), 2021, : 6744 - 6750
[23] Scholarly Text Classification with Sentence BERT and Entity Embeddings
Piao, Guangyuan
TRENDS AND APPLICATIONS IN KNOWLEDGE DISCOVERY AND DATA MINING, PAKDD 2021, 2021, 12705 : 79 - 87
[24] BERT Models for Arabic Text Classification: A Systematic Review
Alammary, Ali Saleh
APPLIED SCIENCES-BASEL, 2022, 12 (11):
[25] Enhancing text classification with attention matrices based on BERT
Yu, Zhiyi
Li, Hong
Feng, Jialin
EXPERT SYSTEMS, 2024, 41 (03)
[26] BVMHA: Text classification model with variable multihead hybrid attention based on BERT
Peng, Bo
Zhang, Tao
Han, Kundong
Zhang, Zhe
Ma, Yuquan
Ma, Mengnan
JOURNAL OF INTELLIGENT & FUZZY SYSTEMS, 2024, 46 (01) : 1443 - 1454
[27] EEBERT: An Emoji-Enhanced BERT Fine-Tuning on Amazon Product Reviews for Text Sentiment Classification
Narejo, Komal Rani
Zan, Hongying
Dharmani, Kheem Parkash
Zhou, Lijuan
Alahmadi, Tahani Jaser
Assam, Muhammad
Sehito, Nabila
Ghadi, Yazeed Yasin
IEEE ACCESS, 2024, 12 : 131954 - 131967
[28] An Application of Transfer Learning: Fine-Tuning BERT for Spam Email Classification
Bhopale, Amol P.
Tiwari, Ashish
MACHINE LEARNING AND BIG DATA ANALYTICS (PROCEEDINGS OF INTERNATIONAL CONFERENCE ON MACHINE LEARNING AND BIG DATA ANALYTICS (ICMLBDA) 2021), 2022, 256 : 67 - 77
[29] Fine-Tuned BERT Model for Large Scale and Cognitive Classification of MOOCs
Sebbaq, Hanane
El Faddouli, Nour-eddine
INTERNATIONAL REVIEW OF RESEARCH IN OPEN AND DISTRIBUTED LEARNING, 2022, 23 (02): : 170 - 190
[30] Website Category Classification Using Fine-tuned BERT Language Model
Demirkiran, Ferhat
Cayir, Aykut
Unal, Ugur
Dag, Hasan
2020 5TH INTERNATIONAL CONFERENCE ON COMPUTER SCIENCE AND ENGINEERING (UBMK), 2020, : 333 - 336

← 1 2 3 4 5 →