Adaptive Hierarchical Text Classification Using ERNIE and Dynamic Threshold Pruning

被引:0
作者
Chen, Han [1 ]
Zhang, Yangsen [1 ]
Jiang, Yuru [1 ]
Duan, Ruixue [1 ]
机构
[1] Beijing Informat Sci & Technol Univ, Inst Intelligent Informat Proc, Beijing 100101, Peoples R China
来源
IEEE ACCESS | 2024年 / 12卷
基金
中国国家自然科学基金;
关键词
Hierarchical text classification; data augmentation; large language model; graph attention network; ERNIE;
D O I
10.1109/ACCESS.2024.3519954
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
Hierarchical Text Classification (HTC) is a challenging task where labels are structured in a tree or Directed Acyclic Graph (DAG) format. Current approaches often struggle with data imbalance and fail to fully capture rich semantic information. This paper proposes an Adaptive Hierarchical Text Classification method, EDTPA (ERNIE and Dynamic Threshold Pruning-based Adaptive classification), which leverages Large Language Models (LLMs) for data augmentation to mitigate imbalanced datasets. The model first uses Graph Attention Networks (GAT) to capture hierarchical dependencies among labels, effectively modeling structured relationships. ERNIE enhances the semantic representation of both the text and hierarchical labels, optimizing the model's ability to process Chinese text. An attention mechanism strengthens the alignment between text and labels, improving accuracy. The model combines global and local information flows, while dynamic threshold pruning prunes low-probability branches, improving interpretability. Results on the Chinese Scientific Literature (CSL) dataset show EDTPA significantly outperforms baseline models in both Micro-F1 and Macro-F1 scores, effectively addressing data imbalance and improving classification performance.
引用
收藏
页码:193641 / 193652
页数:12
相关论文
共 35 条
[1]  
Wang Z., Wang P., Liu T., Lin B., Cao Y., Sui Z., Wang H., HPT: Hierarchy-aware prompt tuning for hierarchical text classification, (2022)
[2]  
Rojas K.R., Bustamante G., Oncevay A., Cabezudo M.A.S., Efficient strategies for hierarchical text classification: External knowledge and auxiliary tasks, (2020)
[3]  
Chen H., Ma Q., Lin Z., Yan J., Hierarchy-aware label semantics matching network for hierarchical text classification, Proc. 59th Annu. Meeting Assoc. Comput. Linguistics 11th Int. Joint Conf. Natural Lang. Process., pp. 4370-4379, (2021)
[4]  
Mullick S.S., Datta S., Das S., Generative adversarial minority oversampling, Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), pp. 1695-1704, (2019)
[5]  
Kim J., Jeong J., Shin J., Imbalanced classification via adversarial minority over-sampling, Proc. Int. Conf. Learn. Represent., pp. 1-10, (2019)
[6]  
Ling C.X., Sheng V.S., Cost-sensitive learning and the class imbalance problem, Encyclopedia of Machine Learning, pp. 231-235, (2008)
[7]  
Parambath S.P., Usunier N., Grandvalet Y., Optimizing F-measures by cost-sensitive classification, Proc. Adv. Neural Inf. Process. Syst., 27, pp. 1-9, (2014)
[8]  
Krawczyk B., Learning from imbalanced data: Open challenges and future directions, Prog. Artif. Intell., 5, 4, pp. 221-232, (2016)
[9]  
Brown T.B., Et al., Language models are few-shot learners, Proc. Adv. Neural Inf. Process. Syst., pp. 1877-1901, (2020)
[10]  
Achiam J., GPT-4 technical report, (2023)