Convolutional Neural Network Bottleneck Features for bi-directional Generalized Variable Parameter HMMs

Recently, convolutional neural networks (CNNs) have been applied successfully to acoustic modelling in speech recognition. As the bottleneck features from CNNs contain inherently discriminative and rich context information, the standard approach is to augment the conventional acoustic features with the CNN bottleneck features in a tandem framework. To better capture the highly complex relationship between them, a novel bidirectional generalized variable parameter HMM (GVP-HMM) based approach is proposed in this paper. In this approach, the trajectories of continuous acoustic features space HMM parameters, as well as the model space linear transforms against CNN bottleneck features are modelled by polynomial functions. The optimal GVP-HMM model structure for each direction, which is determined by the locally varying polynomial parameters and degrees, can be automatically learnt using model selection techniques. The proposed bi-directional GVP-HMM based approach gave a word error rate of 12.22% on the Aurora 4 task. In particular, a significant error rate reduction of 18.09% relative was obtained over the baseline tandem HMM system using CNN bottleneck features on the secondary microphone channel condition.