Shared-Distribution Hidden Markov Models for Speech Recognition

Parameter sharing plays an important role in statistical modeling since the amount of training data is usually limited. On one hand, we would like to use models that are as detailed as possible. On the other hand, with models that are too detailed, we can no longer reliably estimate the parameters. As a parameter-sharing technique, generalized triphones may force two models to be merged when only parts of the models are similar. This problem can be avoided if sharing is carried out at a sub-model level. In this paper, a shared-distribution hidden Markov model is presented for speaker-independent continuous speech recognition. Here, the output distributions across different phonetic HMM's are shared with each other when they exhibit acoustic similarity. This sharing also gives us the freedom to use a larger number of Markov states for each phonetic model. Although an increase in the number of states will increase the total number of free parameters, with distribution sharing we can collapse redundant states while maintaining necessary ones. The shared-distribution model reduced the word error rate on the DARPA Resource Management task by 20% in comparison with the generalized-triphone model.