Automated Speech Production Assessment of Hard of Hearing Children

A new method for the automated speech production assessment (ASPA) of hearing impaired children is presented in this paper, providing feedback about the pronunciation quality of words and sentences uttered during unsupervised practice in the course of speech development. A database of the sounds produced by hearing impaired subjects was set up and assessed with a subjective test. The Mean Opinion Score (MOS) obtained in this way constituted the reference for automated assessment. The essence of the ASPA method is the joint assessment of sound and rhythm errors. After several methods were tested, the output activity of the neural networks trained to classify speech sounds was used to assess sound correctness. Dynamic time warping, adapted to the speech of the hearing impaired, was used to determine rhythm errors. ASPA provides input data for an expert system for the selection of the next word to be practiced. The novelty of the procedure is that it provides a method for the assessment of non-typifiable pronunciation errors. Results were compared with individual expert assessment and subjective tests. Automated assessment surpassed the overwhelming majority of subjective assessors and approximated the correctness of individual expert assessment. Our ASPA method is implemented in our "Speech Assistant" application, which also provides a language-independent sound visualization module and is successfully applied to assist the hearing impaired.