IMPROVING THE INTERPRETABILITY OF DEEP NEURAL NETWORKS WITH STIMULATED LEARNING

Deep Neural Networks (DNNs) have demonstrated improvements in acoustic modelling for automatic speech recognition. However, they are often used as a black box, and not much is understood about what each of the hidden layers does. We seek to understand how the activations in the hidden layers change with different input, and how we can leverage such knowledge to modify the behaviour of the model. To this end, we propose stimulated deep learning where stimuli are introduced during the DNN training process to influence the behaviour of the hidden units. Specifically, constraints are applied so that the hidden units of each layer will exhibit phone-dependent regional activities when arranged in a 2-dimensional grid. We demonstrate that such constraints are able to yield visible activation regions without compromising the classification of the network and suppressing the activations for a region affects the classification accuracy of the corresponding phone more than the others.