Sentiment Analysis: Effect of Combining BERT as an Embedding Technique with CNN Model for Tunisian Dialect

In this paper, we present an enhanced BERT methodology for sentiment classification of a Tunisian corpus. We introduce a Tunisian optimized BERT model, named TunRoBERTa, which surpasses the performance of Multilingual-BERT, CNN, CNN combined with LSTM, and RoBERTa. Additionally, we incorporate TunRoBERTa as an embedding technique with Convolutional Neural Networks (CNN). The experimental results demonstrate that the combination of TunRoBERTa and CNN yields the highest performance compared to the previous models. Our findings outperform Multilingual-BERT, CNN, and CNN combined with LSTM.