A high-performance speech neuroprosthesis

被引:137
|
作者
Willett, Francis R. [1 ]
Kunz, Erin M. [2 ,3 ]
Fan, Chaofei [4 ]
Avansino, Donald T. [1 ]
Wilson, Guy H. [5 ]
Choi, Eun Young [6 ]
Kamdar, Foram [6 ]
Glasser, Matthew F. [7 ,8 ]
Hochberg, Leigh R. [9 ,10 ,11 ,12 ]
Druckmann, Shaul [13 ]
Shenoy, Krishna V. [1 ,2 ,3 ,13 ,14 ,15 ]
Henderson, Jaimie M. [3 ,6 ]
机构
[1] Stanford Univ, Howard Hughes Med Inst, Stanford, CA 94305 USA
[2] Stanford Univ, Dept Elect Engn, Stanford, CA 94305 USA
[3] Stanford Univ, Wu Tsai Neurosci Inst, Stanford, CA USA
[4] Stanford Univ, Dept Comp Sci, Stanford, CA 94305 USA
[5] Stanford Univ, Dept Neurosci, Stanford, CA 94305 USA
[6] Stanford Univ, Dept Neurosurg, Stanford, CA USA
[7] Washington Univ St Louis, Dept Neurosci, St Louis, MO USA
[8] Washington Univ St Louis, Dept Radiol, St Louis, MO USA
[9] Providence VA Med Ctr, VA RR&D Ctr Neurorestorat & Neurotechnol Rehabil, Providence, RI USA
[10] Brown Univ, Sch Engn, Providence, RI USA
[11] Brown Univ, Carney Inst Brain Sci, Providence, RI USA
[12] Harvard Med Sch, Massachusetts Gen Hosp, Dept Neurol, Ctr Neurotechnol & Neurorecovery, Boston, MA USA
[13] Stanford Univ, Dept Neurobiol, Stanford, CA USA
[14] Stanford Univ, Dept Bioengn, Stanford, CA USA
[15] Stanford Univ, Biox Program, Stanford, CA USA
关键词
BROCAS AREA; LANGUAGE; REGIONS; CORTEX;
D O I
10.1038/s41586-023-06377-x
中图分类号
O [数理科学和化学]; P [天文学、地球科学]; Q [生物科学]; N [自然科学总论];
学科分类号
07 ; 0710 ; 09 ;
摘要
Speech brain-computer interfaces (BCIs) have the potential to restore rapid communication to people with paralysis by decoding neural activity evoked by attempted speech into text(1,2) or sound(3,4). Early demonstrations, although promising, have not yet achieved accuracies sufficiently high for communication of unconstrained sentences from a large vocabulary(1-7). Here we demonstrate a speech-to-text BCI that records spiking activity from intracortical microelectrode arrays. Enabled by these high-resolution recordings, our study participant-who can no longer speak intelligibly owing to amyotrophic lateral sclerosis-achieved a 9.1% word error rate on a 50-word vocabulary (2.7times fewer errors than the previous state-of-the-art speech BCI2) and a 23.8% word error rate on a 125,000-word vocabulary (the first successful demonstration, to our knowledge, of large-vocabulary decoding). Our participant's attempted speech was decoded at 62words per minute, which is 3.4times as fast as the previous record(8) and begins to approach the speed of natural conversation (160words per minute(9)). Finally, we highlight two aspects of the neural code for speech that are encouraging for speech BCIs: spatially intermixed tuning to speech articulators that makes accurate decoding possible from only a small region of cortex, and a detailed articulatory representation of phonemes that persists years after paralysis. These results show a feasible path forward for restoring rapid communication to people with paralysis who can no longer speak.
引用
收藏
页码:1031 / 1036
页数:18
相关论文
共 50 条
  • [1] A high-performance speech neuroprosthesis
    Francis R. Willett
    Erin M. Kunz
    Chaofei Fan
    Donald T. Avansino
    Guy H. Wilson
    Eun Young Choi
    Foram Kamdar
    Matthew F. Glasser
    Leigh R. Hochberg
    Shaul Druckmann
    Krishna V. Shenoy
    Jaimie M. Henderson
    Nature, 2023, 620 : 1031 - 1036
  • [2] A high-performance neuroprosthesis for speech decoding and avatar control
    Metzger, Sean L.
    Littlejohn, Kaylo T.
    Silva, Alexander B.
    Moses, David A.
    Seaton, Margaret P.
    Wang, Ran
    Dougherty, Maximilian E.
    Liu, Jessie R.
    Wu, Peter
    Berger, Michael A.
    Zhuravleva, Inga
    Tu-Chan, Adelyn
    Ganguly, Karunesh
    Anumanchipalli, Gopala K.
    Chang, Edward F.
    NATURE, 2023, 620 (7976) : 1037 - +
  • [3] A high-performance neuroprosthesis for speech decoding and avatar control (vol 620, pg 1037, 2024)
    Metzger, Sean L.
    Littlejohn, Kaylo T.
    Silva, Alexander B.
    Moses, David A.
    Seaton, Margaret P.
    Wang, Ran
    Dougherty, Maximilian E.
    Liu, Jessie R.
    Wu, Peter
    Berger, Michael A.
    Zhuravleva, Inga
    Tu-Chan, Adelyn
    Ganguly, Karunesh
    Anumanchipalli, Gopala K.
    Chang, Edward F.
    NATURE, 2024, 631 (8021) : E13 - E13
  • [4] The speech neuroprosthesis
    Silva, Alexander B.
    Littlejohn, Kaylo T.
    Liu, Jessie R.
    Moses, David A.
    Chang, Edward F.
    NATURE REVIEWS NEUROSCIENCE, 2024, 25 (07) : 473 - 492
  • [5] Editorial: The intersection of artificial intelligence and brain for high-performance neuroprosthesis and cyborg systems
    Qi, Yu
    Sun, Yu
    Liu, Quanying
    Zhang, Qiaosheng
    Cai, Hanshu
    Zheng, Qian
    FRONTIERS IN NEUROSCIENCE, 2023, 17
  • [6] Toward a Speech Neuroprosthesis
    Chang, Edward F.
    Anumanchipalli, Gopala K.
    JAMA-JOURNAL OF THE AMERICAN MEDICAL ASSOCIATION, 2020, 323 (05): : 413 - 414
  • [7] Neuromorphic computing facilitates deep brain-machine fusion for high-performance neuroprosthesis
    Qi, Yu
    Chen, Jiajun
    Wang, Yueming
    FRONTIERS IN NEUROSCIENCE, 2023, 17
  • [8] A high-performance psychoacoustics approach to speech quality evaluation
    Network Laboratory, School of Electronics and Information Engineering, Xi'an Jiaotong University, Xi'an, Shaanxi, 710049, China
    不详
    Inf. Technol. J., 2006, 3 (485-488):
  • [9] IMPROVED HMM MODELS FOR HIGH-PERFORMANCE SPEECH RECOGNITION
    AUSTIN, S
    BARRY, C
    CHOW, YL
    DERR, A
    KIMBALL, O
    KUBALA, F
    MAKHOUL, J
    PLACEWAY, P
    RUSSELL, W
    SCHWARTZ, R
    YU, G
    SPEECH AND NATURAL LANGUAGE, 1989, : 249 - 255
  • [10] HIGH-PERFORMANCE PROCESS FITS SPEECH SYSTEM ON A CHIP
    HARA, H
    TORII, S
    TOBA, Y
    NOMIYA, K
    ELECTRONICS, 1981, 54 (20): : 127 - 129