CONTROLLING EYE-MOVEMENTS WITH HIDDEN MARKOV-MODELS

Advances in technology and in active vision research allow and encourage sequential visual information acquisition. Hidden Markov models (HMMs) can represent probabilistic sequences and probabilistic graph structures: here we explore their use in controlling the acquisition of visual information. We include a brief tutorial with two examples: (1) use input sequences to derive an aspect graph and (2) similarly derive a finite state machine for control of visual processing. The first main topic is the use of HMMs in both their learning and generative modes, and their augmentation to allow inputs sensed during generation to modify the generated outputs temporarily or permanently. We propose these augmented HMMs as a theory of adaptive skill acquisition and generation. The second main topic builds on the first: the augmented HMMs can be used for knowledge fusion. We give an example, the what-where-AHMM, which creates a hybrid skill from separate skills based on object location and object identity. Insofar as low-level skills can be learned from the output of high-level cognitive processes, AHMMs can provide a link between high-level and low-level vision.