A high-performance speech neuroprosthesis

被引:180
作者
Willett, Francis R. [1 ]
Kunz, Erin M. [2 ,3 ]
Fan, Chaofei [4 ]
Avansino, Donald T. [1 ]
Wilson, Guy H. [5 ]
Choi, Eun Young [6 ]
Kamdar, Foram [6 ]
Glasser, Matthew F. [7 ,8 ]
Hochberg, Leigh R. [9 ,10 ,11 ,12 ]
Druckmann, Shaul [13 ]
Shenoy, Krishna V. [1 ,2 ,3 ,13 ,14 ,15 ]
Henderson, Jaimie M. [3 ,6 ]
机构
[1] Stanford Univ, Howard Hughes Med Inst, Stanford, CA 94305 USA
[2] Stanford Univ, Dept Elect Engn, Stanford, CA 94305 USA
[3] Stanford Univ, Wu Tsai Neurosci Inst, Stanford, CA USA
[4] Stanford Univ, Dept Comp Sci, Stanford, CA 94305 USA
[5] Stanford Univ, Dept Neurosci, Stanford, CA 94305 USA
[6] Stanford Univ, Dept Neurosurg, Stanford, CA USA
[7] Washington Univ St Louis, Dept Neurosci, St Louis, MO USA
[8] Washington Univ St Louis, Dept Radiol, St Louis, MO USA
[9] Providence VA Med Ctr, VA RR&D Ctr Neurorestorat & Neurotechnol Rehabil, Providence, RI USA
[10] Brown Univ, Sch Engn, Providence, RI USA
[11] Brown Univ, Carney Inst Brain Sci, Providence, RI USA
[12] Harvard Med Sch, Massachusetts Gen Hosp, Dept Neurol, Ctr Neurotechnol & Neurorecovery, Boston, MA USA
[13] Stanford Univ, Dept Neurobiol, Stanford, CA USA
[14] Stanford Univ, Dept Bioengn, Stanford, CA USA
[15] Stanford Univ, Biox Program, Stanford, CA USA
关键词
BROCAS AREA; LANGUAGE; REGIONS; CORTEX;
D O I
10.1038/s41586-023-06377-x
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text(1,2) or sound(3,4). Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary(1-7). Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant-who can no longer speak intelligibly owing to amyotrophic lateral sclerosis-achieved a 9.1% word error rate on a 50-word vocabulary (2.7times fewer errors than the previous state-of-the-art speech BCI2) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant's attempted speech was decoded at 62words per minute, which is 3.4times as fast as the previous record(8) and begins to approach the speed of natural conversation (160words per minute(9)). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.
引用
收藏
页码:1031 / 1036
页数:18
相关论文
共 38 条
[1]   Dissociation of Broca?s area from Broca?s aphasia in patients undergoing neurosurgical resections [J].
Andrews, John P. ;
Cahn, Nathan ;
Speidel, Benjamin A. ;
Chung, Jason E. ;
Levy, Deborah F. ;
Wilson, Stephen M. ;
Chang, Edward F. ;
Berger, Mitchel S. .
JOURNAL OF NEUROSURGERY, 2023, 138 (03) :847-857
[2]  
[Anonymous], 2010, A Phonetic Examination of California
[3]   Speech synthesis from neural decoding of spoken sentences [J].
Anumanchipalli, Gopala K. ;
Chartier, Josh ;
Chang, Edward F. .
NATURE, 2019, 568 (7753) :493-+
[4]   How Localized are Language Brain Areas? A Review of Brodmann Areas Involvement in Oral Language [J].
Ardila, Alfredo ;
Bernal, Byron ;
Rosselli, Monica .
ARCHIVES OF CLINICAL NEUROPSYCHOLOGY, 2016, 31 (01) :112-122
[5]   Functional organization of human sensorimotor cortex for speech articulation [J].
Bouchard, Kristofer E. ;
Mesgarani, Nima ;
Johnson, Keith ;
Chang, Edward F. .
NATURE, 2013, 495 (7441) :327-332
[6]  
Broca P., 1861, Bulletins de la Societe d'anatomie, V6, P398
[7]   Stabilization of a brain-computer interface via the alignment of low-dimensional spaces of neural activity [J].
Degenhart, Alan D. ;
Bishop, William E. ;
Oby, Emily R. ;
Tyler-Kabara, Elizabeth C. ;
Chase, Steven M. ;
Batista, Aaron P. ;
Yu, Byron M. .
NATURE BIOMEDICAL ENGINEERING, 2020, 4 (07) :672-685
[8]   A cryptography-based approach for movement decoding [J].
Dyer, Eva L. ;
Azar, Mohammad Gheshlaghi ;
Perich, Matthew G. ;
Fernandes, Hugo L. ;
Naufel, Stephanie ;
Miller, Lee E. ;
Kording, Konrad P. .
NATURE BIOMEDICAL ENGINEERING, 2017, 1 (12) :967-+
[9]  
Farshchian A., 2019, PREPRINT, DOI [10.48550/arXiv.1810.00045, DOI 10.48550/ARXIV.1810.00045]
[10]   Redefining the role of Broca's area in speech [J].
Flinker, Adeen ;
Korzeniewska, Anna ;
Shestyuk, Avgusta Y. ;
Franaszczuk, Piotr J. ;
Dronkers, Nina F. ;
Knight, Robert T. ;
Crone, Nathan E. .
PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA, 2015, 112 (09) :2871-2875