Formant-based Frequency Warping for Improving Speaker Adaptation in HMM TTS

Vocal Tract Length Normalization (VLTN), usually implemented as a frequency warping procedure (e.g. bilinear transformation), has been used successfully to adapt the spectral characteristics to a target speaker in speech recognition. In this study we exploit the same concept of frequency warping but concentrate explicitly on mapping the first four formant frequencies of 5 long vowels from source and target speakers. A universal warping function is thus constructed for improving MLLR-based speaker adaptation performance in TTS. The function first warps the frequency scale of the source speaker's speech data toward that of the target speaker and an HMM of the warped features is trained. Finally, MLLR-based speaker adaptation is applied to the trained HMM for synthesizing the target speaker's speech. When tested on a database of 4,000 sentences (source speaker) and 100 sentences of a male and a female speaker (target speakers), the formant based frequency warping has been found very effective in reducing the objective, log spectral distortion over the system without formant frequency warping. The improvement is also subjectively confirmed in AB preference and ABX speaker similarity listening tests.