Perception based method for the investigation of audiovisual integration of speech

被引:11
作者
Huhn, Zsofia [1 ,2 ]
Szirtes, Gabor [3 ]
Lorincz, Andras
Csepe, Valeria [2 ,3 ]
机构
[1] Hungarian Acad Sci, Dept Biophys, KFKI Res Inst Particle & Nucl Phys, H-1121 Budapest, Hungary
[2] Hungarian Acad Sci, Dept Psychophysiol, Res Inst Psychol, H-1121 Budapest, Hungary
[3] Eotvos Lorand Univ, Dept Cognit Psychol, Inst Psychol, H-1364 Budapest, Hungary
关键词
Audiovisual integration; Speech; Multisensory interaction; McGurk effect;
D O I
10.1016/j.neulet.2009.08.077
中图分类号
Q189 [神经科学];
学科分类号
071006 ;
摘要
Speech comprehension is significantly improved by visual input oil the speaker's mouth movements. Audiovisual integration underlying this phenomenon is often studied in EEG experiments in which the event related brain potential (ERP) elicited by a bimodal stimulus is compared to the sum of ERPs triggered by auditory and visual signals of the same source. However, this method leads to spurious results in time ranges when ERP components common to all these stimulus types are present. A method that aims to filter out such common early anticipatory potentials is data high-pass filtering. In the present study, first, we demonstrated that subtle changes in filter cut-off frequency lead to remarkably different results on the interaction effect so that no reliable conclusion on the spatial distribution of the interaction could be drawn. Second, we suggested a different approach for the investigation of ERP correlates of audiovisual integration: bimodal syllables modified by light temporal asynchrony were presented to subjects and ERPs correlating with the fused and unfused perceptions were compared. We found that components corresponding to both auditory N1 and P2 waves were smaller in case of the fused perception, supporting the view that NI and P2 generator activities are suppressed during multimodal speech perception. The N1 effect showed a clearly right hemisphere dominance while the effect around the P2 peak was most pronounced on centroparietal electrodes and dominated over the left hemisphere. (C) 2009 Elsevier Ireland Ltd. All rights reserved
引用
收藏
页码:204 / 209
页数:6
相关论文
共 18 条
  • [1] Bisensory augmentation: A speechreading advantage when speech is clearly audible and intact
    Arnold, P
    Hill, F
    [J]. BRITISH JOURNAL OF PSYCHOLOGY, 2001, 92 : 339 - 355
  • [2] THE SPATIOTEMPORAL ORGANIZATION OF AUDITORY, VISUAL AND AUDITORY-VISUAL EVOKED-POTENTIALS IN RAT CORTEX
    BARTH, DS
    GOLDBERG, N
    BRETT, B
    DI, S
    [J]. BRAIN RESEARCH, 1995, 678 (1-2) : 177 - 190
  • [3] Bimodal speech: early suppressive visual effects in human auditory cortex
    Besle, J
    Fort, A
    Delpuech, C
    Giard, MH
    [J]. EUROPEAN JOURNAL OF NEUROSCIENCE, 2004, 20 (08) : 2225 - 2234
  • [4] Besle J., 2004, Cogn Process, V5, P189, DOI DOI 10.1007/S10339-004-0026-Y
  • [5] Visual Activation and Audiovisual Interactions in the Auditory Cortex during Speech Perception: Intracranial Recordings in Humans
    Besle, Julien
    Fischer, Catherine
    Bidet-Caulet, Aurelie
    Lecaignard, Francoise
    Bertrand, Olivier
    Giard, Marie-Helene
    [J]. JOURNAL OF NEUROSCIENCE, 2008, 28 (52) : 14301 - 14310
  • [6] AN ALTERNATIVE METHOD FOR SIGNIFICANCE TESTING OF WAVE-FORM DIFFERENCE POTENTIALS
    BLAIR, RC
    KARNISKI, W
    [J]. PSYCHOPHYSIOLOGY, 1993, 30 (05) : 518 - 524
  • [7] Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex
    Calvert, GA
    Campbell, R
    Brammer, MJ
    [J]. CURRENT BIOLOGY, 2000, 10 (11) : 649 - 657
  • [8] Cohen J., 1988, Statistical power analysis for the behavioural sciences, V2nd
  • [9] Asynchrony from synchrony: long-range gamma-band neural synchrony accompanies perception of audiovisual speech asynchrony
    Doesburg, Sam M.
    Emberson, Lauren L.
    Rahi, Alan
    Cameron, David
    Ward, Lawrence M.
    [J]. EXPERIMENTAL BRAIN RESEARCH, 2008, 185 (01) : 11 - 20
  • [10] The use of visible speech cues for improving auditory detection of spoken sentences
    Grant, KW
    Seitz, PF
    [J]. JOURNAL OF THE ACOUSTICAL SOCIETY OF AMERICA, 2000, 108 (03) : 1197 - 1208