A Multilingual Sentiment Analysis Model Based on Continual Learning

被引：0

作者：

Zhao, Jiayi ^{[1
]}

Xu, Yuemei ^{[1
]}

Gu, Hanwen ^{[1
]}

机构：

[1] School of Information Science and Technology, Beijing Foreign Studies University, Beijing

来源：

Data Analysis and Knowledge Discovery | 2024年 / 8卷 / 10期

关键词：

Catastrophic Forgetting; Continual Learning; Multilingual Sentiment Analysis;

D O I：

10.11925/infotech.2096-3467.2023.0714

中图分类号：

学科分类号：

摘要：

[Objective] This study addresses the performance degradation due to catastrophic forgetting when multilingual models handle tasks in new languages. [Methods] We proposed a multilingual sentiment analysis model, mLMs-EWC, based on continual learning. The model incorporates continual learning into multilingual models, enabling it to learn new language features while retaining the linguistic characteristics of previously learned languages. [Results] In continual sentiment analysis experiments involving three languages, the mLMs-EWC model outperformed the Multi-BERT model by approximately 5.0% in French and 4.5% in English tasks. Additionally, the mLMs-EWC model was evaluated on a lightweight distilled model, showing an improvement of up to 24.7% in English tasks. [Limitations] This study focuses on three widely used languages, and further validation is needed to assess the model's generalization capability to other languages. [Conclusions] The proposed model can alleviate catastrophic forgetting in multilingual sentiment analysis tasks and achieve continual learning on multilingual datasets. © 2024 Chinese Academy of Sciences. All rights reserved.

引用

页码：44 / 53

页数：9

共 41 条

[1]

Poria S, Hazarika D, Majumder N, Et al., Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research, IEEE Transactions on Affective Computing, 14, 1, pp. 108-132, (2023)

[2]

Joshi P, Santy S, Budhiraja A, Et al., The State and Fate of Linguistic Diversity and Inclusion in the NLP World, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6282-6293, (2020)

[3]

Xu Y M, Cao H, Du W Z, Et al., A Survey of Cross-Lingual Sentiment Analysis: Methodologies, Models and Evaluations, Data Science and Engineering, 7, 3, pp. 279-299, (2022)

[4]

Abdalla M, Hirst G., Cross-Lingual Sentiment Analysis Without (Good) Translation, Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 506-515, (2017)

[5]

Chen X L, Sun Y, Athiwaratkun B, Et al., Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification, Transactions of the Association for Computational Linguistics, 6, pp. 557-570, (2018)

[6]

Devlin J, Chang M W, Lee K, Et al., BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, (2019)

[7]

Yang Z L, Dai Z H, Yang Y M, Et al., XLNet: Generalized Autoregressive Pretraining for Language Understanding, Proceedings of the 33rd International Conference on Neural Information Processing Systems, pp. 5753-5763, (2019)

[8]

Wu S J, Dredze M., Are All Languages Created Equal in Multilingual BERT? [C], Proceedings of the 5th Workshop on Representation Learning for NLP, pp. 120-130, (2020)

[9]

Vu T, Barua A, Lester B, Et al., Overcoming Catastrophic Forgetting in Zero-Shot Cross-Lingual Generation, Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 9279-9300, (2022)

[10]

Vander Eeckt S, Van Hamme H., Using Adapters to Overcome Catastrophic Forgetting in End-to-End Automatic Speech Recognition, Proceedings of 2023 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1-5, (2023)

← 1 2 3 4 5 →