Dysarthria severity classification using multi-head attention and multi-task learning

Identifying the severity of dysarthria is considered a diagnostic step in monitoring the patient's progress and a beneficial step in the transcription of dysarthric speech. In this paper, the effectiveness of using the multi-head attention mechanism (MHA) and the multi-task learning approach is explored for automated dysarthria severity level classification. Dysarthric speech utterances are represented by mel spectrograms and fed to a residual convolutional neural network for effective feature learning. Then the MHA module is added to identify the salient severity-highlighting periods. At the classification end, gender, age, and disorder-type identifications are employed as auxiliary tasks to share mutual information and leverage the severity classification. The performance of the proposed method is evaluated on the Universal Access Speech database. By giving a gain of 11.51% classification accuracy over the baseline system under the speaker-dependent scenario and 11.58% under the speaker-independent scenario, the proposed system demonstrates its potential for the dysarthria severity classification.