Crossmodal binding: Evaluating the "unity assumption" using audiovisual speech stimuli

被引:228
作者
Vatakis, Argiro [1 ]
Spence, Charles [1 ]
机构
[1] Univ Oxford, Dept Expt Psychol, Oxford OX1 3UD, England
来源
PERCEPTION & PSYCHOPHYSICS | 2007年 / 69卷 / 05期
关键词
D O I
10.3758/BF03193776
中图分类号
B84 [心理学];
学科分类号
04 ; 0402 ;
摘要
We investigated whether the "unity assumption" according to which an observer assumes that two different sensory signals refer to the same underlying multisensory event, influences the multisensory integration of audiovisual speech stimuli. Syllables (Experiments 1, 3, and 4) or words (Experiment 2) were presented to participants at a range of different stimulus onset asynchronies using the method of constant stimuli. Participants made unspeeded temporal order judgments regarding which stream (either auditory or visual) had been presented first. The auditory and visual speech stimuli in Experiments 1-3 were either gender matched (i.e., a female face presented together with a female voice) or else gender mismatched (i.e., a female face presented together with a male voice). In Experiment 4, different utterances from the same female speaker were used to generate the matched and mismatched speech video clips. Measuring in terms of the just noticeable difference the participants in all four experiments found it easier to judge which sensory modality had been presented first when evaluating mismatched stimuli than when evaluating the matched-speech stimuli. These results therefore provide the first empirical support for the "unity assumption" in the domain of the multisensory temporal integration of audiovisual speech stimuli.
引用
收藏
页码:744 / 756
页数:13
相关论文
共 70 条
[1]  
ABRY C, 1994, CAH PSYCHOL COGN, V13, P52
[2]   The ventriloquist effect results from near-optimal bimodal integration [J].
Alais, D ;
Burr, D .
CURRENT BIOLOGY, 2004, 14 (03) :257-262
[3]   Audiovisual integration of speech falters under high attention demands [J].
Alsius, A ;
Navarra, J ;
Campbell, R ;
Soto-Faraco, S .
CURRENT BIOLOGY, 2005, 15 (09) :839-843
[4]  
[Anonymous], 2007, PSYCHOL BEAUTY KANSE
[5]   Projecting sensations to external objects: evidence from skin conductance response [J].
Armel, KC ;
Ramachandran, VS .
PROCEEDINGS OF THE ROYAL SOCIETY B-BIOLOGICAL SCIENCES, 2003, 270 (1523) :1499-1506
[6]   Timing sight and sound [J].
Arnold, DH ;
Johnston, A ;
Nishida, S .
VISION RESEARCH, 2005, 45 (10) :1275-1284
[7]   Bayesian integration of visual and auditory signals for spatial localization [J].
Battaglia, PW ;
Jacobs, RA ;
Aslin, RN .
JOURNAL OF THE OPTICAL SOCIETY OF AMERICA A-OPTICS IMAGE SCIENCE AND VISION, 2003, 20 (07) :1391-1397
[8]  
BEDFORD FL, 1994, CAH PSYCHOL COGN, V13, P60
[9]  
Bedford FL, 2001, CAH PSYCHOL COGN, V20, P113
[10]   EFFECT OF DEGREE OF SEPARATION OF VISUAL-AUDITORY STIMULUS AND EYE POSITION UPON SPATIAL INTERACTION OF VISION AND AUDITION [J].
BERMANT, RI ;
WELCH, RB .
PERCEPTUAL AND MOTOR SKILLS, 1976, 43 (02) :487-493