An End-to-End Approach to Automatic Speech Assessment for People with Aphasia

Conventionally, automatic assessment of pathological speech involves two main steps: (1) extraction of pathology-specific features; (2) classification or regression of extracted features. Given the great variety of speech and language disorders, feature design is never a straightforward task, and yet it is most critical to the performance of assessment. This paper presents an end-to-end approach to automatic speech assessment for Cantonese-speaking people with aphasia (PWA). The assessment is formulated as a binary classification problem to differentiate PWA with high scores of subjective assessment from those with low scores. The sequence-to-one GRU-RNN and CNN models are applied to realize the end-to-end mapping from speech signals to the classification result. The speech features used for assessment are learned implicitly by the neural network model. Preliminary experimental results show that the end-to-end approach could reach a performance level comparable to conventional two-step approach. The experimental results also suggest that CNN performs better than sequence-to-one GRU-RNN in this specific task.