Enhancement of selective listening by illusory mislocation of speech sounds due to lip-reading

MECHANISMS of human attention allow selective processing of just the relevant events among the many stimuli bombarding our senses(1). Most laboratory studies examine attention within just a single sense, but in the real world many important events are specified multimodally, as in verbal communication. Speech comprises visual lip movements as well as sounds, and lip-reading contributes to speech perception. even for listeners with good hearing, by a process of audiovisual integration(2). Such examples raise the problem of how we coordinate our spatial attention across the sensory modalities, to select sights and sounds from a common source for further processing. Here we show that this problem is alleviated by allowing some cross-modal matching before attentional selection is completed. Cross modal matching can lead to an illusion, whereby sounds are mislocated at their apparent visual source(3); this crossmodal illusion can enhance selective spatial attention to speech sounds.