Auditory stimulus-response modeling with a match-mismatch task

被引:19
作者
de Cheveigne, Alain [1 ,2 ,3 ,7 ]
Slaney, Malcolm [4 ]
Fuglsang, Soren A. [6 ]
Hjortkjaer, Jens [5 ,6 ]
机构
[1] CNRS, UMR 8248, Lab Syst Perceptifs, Paris, France
[2] PSL, Ecole Normale Super, Dept Etud Cognit, Paris, France
[3] UCL Ear Inst, London, England
[4] Google Res, Machine Hearing Grp, Mountain View, CA USA
[5] Tech Univ Denmark, Dept Hlth Technol, Hearing Syst Sect, Lyngby, Denmark
[6] Copenhagen Univ Hosp Hvidovre, Ctr Funct & Diagnost Imaging & Res, Danish Res Ctr Magnet Resonance, Copenhagen, Denmark
[7] ENS, DEC, Audit, 29 Rue Ulm, F-75230 Paris, France
关键词
EEG; MEG; BCI; auditory; decoding; CCA; attention decoding;
D O I
10.1088/1741-2552/abf771
中图分类号
R318 [生物医学工程];
学科分类号
0831 ;
摘要
Objective. An auditory stimulus can be related to the brain response that it evokes by a stimulus-response model fit to the data. This offers insight into perceptual processes within the brain and is also of potential use for devices such as brain computer interfaces (BCIs). The quality of the model can be quantified by measuring the fit with a regression problem, or by applying it to a classification task and measuring its performance. Approach. Here we focus on a match-mismatch (MM) task that entails deciding whether a segment of brain signal matches, via a model, the auditory stimulus that evoked it. Main results. Using these metrics, we describe a range of models of increasing complexity that we compare to methods in the literature, showing state-of-the-art performance. We document in detail one particular implementation, calibrated on a publicly-available database, that can serve as a robust reference to evaluate future developments. Significance. The MM task allows stimulus-response models to be evaluated in the limit of very high model accuracy, making it an attractive alternative to the more commonly used task of auditory attention detection. The MM task does not require class labels, so it is immune to mislabeling, and it is applicable to data recorded in listening scenarios with only one sound source, thus it is cheap to obtain large quantities of training and testing data. Performance metrics from this task, associated with regression accuracy, provide complementary insights into the relation between stimulus and response, as well as information about discriminatory power directly applicable to BCI applications.
引用
收藏
页数:15
相关论文
共 54 条
[1]   Speech comprehension is correlated with temporal response patterns recorded from auditory cortex [J].
Ahissar, E ;
Nagarajan, S ;
Ahissar, M ;
Protopapas, A ;
Mahncke, H ;
Merzenich, MM .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2001, 98 (23) :13367-13372
[2]   Human cortical responses to the speech envelope [J].
Aiken, Steven J. ;
Picton, Terence W. .
EAR AND HEARING, 2008, 29 (02) :139-157
[3]   Towards reconstructing intelligible speech from the human auditory cortex [J].
Akbari, Hassan ;
Khalighinejad, Bahar ;
Herrero, Jose L. ;
Mehta, Ashesh D. ;
Mesgarani, Nima .
SCIENTIFIC REPORTS, 2019, 9 (1)
[4]  
Andrienko G., 2013, P 1 ACM SIGSPATIAL I, P9, DOI [10.1145/2534848.2534851, DOI 10.1145/2534848.2534851]
[5]   Objects that Sound [J].
Arandjelovic, Relja ;
Zisserman, Andrew .
COMPUTER VISION - ECCV 2018, PT I, 2018, 11205 :451-466
[6]   Where is the cocktail party? Decoding locations of attended and unattended moving sound sources using EEG [J].
Bednar, Adam ;
Lalor, Edmund C. .
NEUROIMAGE, 2020, 205
[7]   Auditory-Inspired Speech Envelope Extraction Methods for Improved EEG-Based Auditory Attention Detection in a Cocktail Party Scenario [J].
Biesmans, Wouter ;
Das, Neetha ;
Francart, Tom ;
Bertrand, Alexander .
IEEE TRANSACTIONS ON NEURAL SYSTEMS AND REHABILITATION ENGINEERING, 2017, 25 (05) :402-412
[8]   Semantic Context Enhances the Early Auditory Encoding of Natural Speech [J].
Broderick, Michael P. ;
Anderson, Andrew J. ;
Lalor, Edmund C. .
JOURNAL OF NEUROSCIENCE, 2019, 39 (38) :7564-7575
[9]   Comparison of Two-Talker Attention Decoding from EEG with Nonlinear Neural Networks and Linear Methods [J].
Ciccarelli, Gregory ;
Nolan, Michael ;
Perricone, Joseph ;
Calamia, Paul T. ;
Haro, Stephanie ;
O'Sullivan, James ;
Mesgarani, Nima ;
Quatieri, Thomas F. ;
Smalt, Christopher J. .
SCIENTIFIC REPORTS, 2019, 9 (1)
[10]   The Multivariate Temporal Response Function (mTRF) Toolbox: A MATLAB Toolbox for Relating Neural Signals to Continuous Stimuli [J].
Crosse, Michael J. ;
Di Liberto, Giovanni M. ;
Bednar, Adam ;
Lalor, Edmund C. .
FRONTIERS IN HUMAN NEUROSCIENCE, 2016, 10