Visual analytics tool for the interpretation of hidden states in recurrent neural networks

被引:6
作者
Garcia, Rafael [1 ]
Munz, Tanja [1 ]
Weiskopf, Daniel [1 ]
机构
[1] Univ Stuttgart, VISUS, D-70569 Stuttgart, Germany
关键词
Visual analytics; Visualization; Machine learning; Classification; Recurrent neural networks; Long short-term memory; Hidden states; Interpretability; Natural language processing; Nonlinear projection;
D O I
10.1186/s42492-021-00090-0
中图分类号
TP39 [计算机的应用];
学科分类号
081203 ; 0835 ;
摘要
In this paper, we introduce a visual analytics approach aimed at helping machine learning experts analyze the hidden states of layers in recurrent neural networks. Our technique allows the user to interactively inspect how hidden states store and process information throughout the feeding of an input sequence into the network. The technique can help answer questions, such as which parts of the input data have a higher impact on the prediction and how the model correlates each hidden state configuration with a certain output. Our visual analytics approach comprises several components: First, our input visualization shows the input sequence and how it relates to the output (using color coding). In addition, hidden states are visualized through a nonlinear projection into a 2-D visualization space using t-distributed stochastic neighbor embedding to understand the shape of the space of the hidden states. Trajectories are also employed to show the details of the evolution of the hidden state configurations. Finally, a time-multi-class heatmap matrix visualizes the evolution of the expected predictions for multi-class classifiers, and a histogram indicates the distances between the hidden states within the original space. The different visualizations are shown simultaneously in multiple views and support brushing-and-linking to facilitate the analysis of the classifications and debugging for misclassified input sequences. To demonstrate the capability of our approach, we discuss two typical use cases for long short-term memory models applied to two widely used natural language processing datasets.
引用
收藏
页数:13
相关论文
共 50 条
[41]   Recurrent neural networks with long term temporal dependencies in machine tool wear diagnosis and prognosis [J].
Jianlei Zhang ;
Yukun Zeng ;
Binil Starly .
SN Applied Sciences, 2021, 3
[42]   Recurrent neural networks with long term temporal dependencies in machine tool wear diagnosis and prognosis [J].
Zhang, Jianlei ;
Zeng, Yukun ;
Starly, Binil .
SN APPLIED SCIENCES, 2021, 3 (04)
[43]   A motion model based on recurrent neural networks for visual object tracking [J].
Shahbazi, Mohammad ;
Bayat, Mohammad Hosein ;
Tarvirdizadeh, Bahram .
IMAGE AND VISION COMPUTING, 2022, 126
[44]   Quantum Gated Recurrent Neural Networks [J].
Li, Yanan ;
Wang, Zhimin ;
Xing, Ruipeng ;
Shao, Changheng ;
Shi, Shangshang ;
Li, Jiaxin ;
Zhong, Guoqiang ;
Gu, Yongjian .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2025, 47 (04) :2493-2504
[45]   Activities of Daily Living Classification using Recurrent Neural Networks [J].
Jurca, Roxana ;
Cioara, Tudor ;
Anghel, Ionut ;
Antal, Marcel ;
Pop, Claudia ;
Moldovan, Dorin .
2018 17TH ROEDUNET IEEE INTERNATIONAL CONFERENCE: NETWORKING IN EDUCATION AND RESEARCH (ROEDUNET), 2018,
[46]   Automatic diacritization of Arabic text using recurrent neural networks [J].
Abandah, Gheith A. ;
Graves, Alex ;
Al-Shagoor, Balkees ;
Arabiyat, Alaa ;
Jamour, Fuad ;
Al-Taee, Majid .
INTERNATIONAL JOURNAL ON DOCUMENT ANALYSIS AND RECOGNITION, 2015, 18 (02) :183-197
[47]   Detangler: Visual Analytics for Multiplex Networks [J].
Renoust, B. ;
Melancon, G. ;
Munzner, T. .
COMPUTER GRAPHICS FORUM, 2015, 34 (03) :321-330
[48]   Automatic diacritization of Arabic text using recurrent neural networks [J].
Gheith A. Abandah ;
Alex Graves ;
Balkees Al-Shagoor ;
Alaa Arabiyat ;
Fuad Jamour ;
Majid Al-Taee .
International Journal on Document Analysis and Recognition (IJDAR), 2015, 18 :183-197
[49]   Tasks for Visual Analytics in Multilayer Networks [J].
Zhang Xitao ;
Wu Lingda ;
Hu Huaquan ;
Yu Shaobo .
2017 IEEE SECOND INTERNATIONAL CONFERENCE ON DATA SCIENCE IN CYBERSPACE (DSC), 2017, :368-371
[50]   Review: visual analytics of climate networks [J].
Nocke, T. ;
Buschmann, S. ;
Donges, J. F. ;
Marwan, N. ;
Schulz, H. -J. ;
Tominski, C. .
NONLINEAR PROCESSES IN GEOPHYSICS, 2015, 22 (05) :545-570