Explainable Sentiment Analysis: A Hierarchical Transformer-Based Extractive Summarization Approach

被引:15
作者
Bacco, Luca [1 ,2 ]
Cimino, Andrea [2 ]
Dell'Orletta, Felice [2 ]
Merone, Mario [1 ]
机构
[1] Univ Campus Biomed Roma, Unit Comp Syst & Bioinformat, Dept Engn, I-00128 Rome, Italy
[2] Ist Linguist Computaz Antonio Zampolli ILC CNR, ItaliaNLP Lab, I-56124 Pisa, Italy
关键词
sentiment analysis; explainability; hierarchical transformers; extractive summarization;
D O I
10.3390/electronics10182195
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
In recent years, the explainable artificial intelligence (XAI) paradigm is gaining wide research interest. The natural language processing (NLP) community is also approaching the shift of paradigm: building a suite of models that provide an explanation of the decision on some main task, without affecting the performances. It is not an easy job for sure, especially when very poorly interpretable models are involved, like the almost ubiquitous (at least in the NLP literature of the last years) transformers. Here, we propose two different transformer-based methodologies exploiting the inner hierarchy of the documents to perform a sentiment analysis task while extracting the most important (with regards to the model decision) sentences to build a summary as the explanation of the output. For the first architecture, we placed two transformers in cascade and leveraged the attention weights of the second one to build the summary. For the other architecture, we employed a single transformer to classify the single sentences in the document and then combine the probability scores of each to perform the classification and then build the summary. We compared the two methodologies by using the IMDB dataset, both in terms of classification and explainability performances. To assess the explainability part, we propose two kinds of metrics, based on benchmarking the models' summaries with human annotations. We recruited four independent operators to annotate few documents retrieved from the original dataset. Furthermore, we conducted an ablation study to highlight how implementing some strategies leads to important improvements on the explainability performance of the cascade transformers model.
引用
收藏
页数:19
相关论文
共 63 条
  • [21] DARPA's Explainable Artificial Intelligence Program
    Gunning, David
    Aha, David W.
    [J]. AI MAGAZINE, 2019, 40 (02) : 44 - 58
  • [22] Hewitt J, 2019, 2019 CONFERENCE OF THE NORTH AMERICAN CHAPTER OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS: HUMAN LANGUAGE TECHNOLOGIES (NAACL HLT 2019), VOL. 1, P4129
  • [23] Howard J, 2018, PROCEEDINGS OF THE 56TH ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS (ACL), VOL 1, P328
  • [24] Hu MQ, 2004, PROCEEDING OF THE NINETEENTH NATIONAL CONFERENCE ON ARTIFICIAL INTELLIGENCE AND THE SIXTEENTH CONFERENCE ON INNOVATIVE APPLICATIONS OF ARTIFICIAL INTELLIGENCE, P755
  • [25] Kovaleva O., 2019, Revealing the Dark Secrets of BERT
  • [27] Reliability in content analysis - Some common misconceptions and recommendations
    Krippendorff, K
    [J]. HUMAN COMMUNICATION RESEARCH, 2004, 30 (03) : 411 - 433
  • [28] Krippendorff K., 2018, Content Analysis: An Introduction to Its Methodology
  • [29] Larochelle Hugo, 2010, NEURAL INFORM PROCES, V23
  • [30] Letarte G., 2018, P 2018 EMNLP WORKSH, DOI DOI 10.18653/V1/W18-5429