OPTIMAL-CONTROL FOR TRAINING - THE MISSING LINK BETWEEN HIDDEN MARKOV-MODELS AND CONNECTIONIST NETWORKS

For every Hidden Markov Model there is a set of "forward" probabilities that need to be computed for both the recognition and the training problem. These probabilities are computed recursively and hence the computation can be performed by a multistage, feedforward network that we will call Hidden Markov Model Net (HMMN). This network has exactly the same architecture as the standard Connectionist Network (CN). Furthermore training an Hidden Markov Model is equivalent to optimizing a function of the HMMN; training a CN is equivalent to optimizing a function of the CN. Due to the multistage architecture, both problems can be seen as Optimal Control problems. By applying standard Optimal Control techniques we discover in both problems that certain backpropagated quantities (backward probabilities for HMMN, backward propagated erros for CN) are of crucial importance to the solution. So HMM's and CN's are similar in architecture and training.