Shared Control of Bimanual Robotic Limbs With a Brain-Machine Interface for Self-Feeding

被引:34
作者
Handelman, David A. [1 ]
Osborn, Luke E. [1 ]
Thomas, Tessy M. [2 ]
Badger, Andrew R. [1 ]
Thompson, Margaret [1 ]
Nickl, Robert W. [3 ]
Anaya, Manuel A. [3 ]
Wormley, Jared M. [1 ]
Cantarero, Gabriela L. [3 ]
McMullen, David [4 ]
Crone, Nathan E. [5 ]
Wester, Brock [1 ]
Celnik, Pablo A. [3 ]
Fifer, Matthew S. [1 ]
Tenore, Francesco V. [1 ]
机构
[1] Johns Hopkins Univ, Dept Res & Exploratory Dev, Appl Phys Lab, Laurel, MD 20723 USA
[2] Johns Hopkins Univ, Dept Biomed Engn, Sch Med, Baltimore, MD USA
[3] Johns Hopkins Med, Dept Phys Med & Rehabil, Baltimore, MD USA
[4] Natl Inst Mental Hlth, NIH, Bethesda, MD USA
[5] Johns Hopkins Med, Dept Neurol, Baltimore, MD USA
基金
美国国家卫生研究院;
关键词
human machine teaming; brain computer interface (BCI); bimanual control; robotic shared control; activities of daily living (ADL); STIMULATION; SENSATIONS;
D O I
10.3389/fnbot.2022.918001
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Advances in intelligent robotic systems and brain-machine interfaces (BMI) have helped restore functionality and independence to individuals living with sensorimotor deficits; however, tasks requiring bimanual coordination and fine manipulation continue to remain unsolved given the technical complexity of controlling multiple degrees of freedom (DOF) across multiple limbs in a coordinated way through a user input. To address this challenge, we implemented a collaborative shared control strategy to manipulate and coordinate two Modular Prosthetic Limbs (MPL) for performing a bimanual self-feeding task. A human participant with microelectrode arrays in sensorimotor brain regions provided commands to both MPLs to perform the self-feeding task, which included bimanual cutting. Motor commands were decoded from bilateral neural signals to control up to two DOFs on each MPL at a time. The shared control strategy enabled the participant to map his four-DOF control inputs, two per hand, to as many as 12 DOFs for specifying robot end effector position and orientation. Using neurally-driven shared control, the participant successfully and simultaneously controlled movements of both robotic limbs to cut and eat food in a complex bimanual self-feeding task. This demonstration of bimanual robotic system control via a BMI in collaboration with intelligent robot behavior has major implications for restoring complex movement behaviors for those living with sensorimotor deficits.
引用
收藏
页数:10
相关论文
共 33 条
[1]   Decoding motor imagery from the posterior parietal cortex of a tetraplegic human [J].
Aflalo, Tyson ;
Kellis, Spencer ;
Klaes, Christian ;
Lee, Brian ;
Shi, Ying ;
Pejsa, Kelsie ;
Shanfield, Kathleen ;
Hayes-Jackson, Stephanie ;
Aisen, Mindy ;
Heck, Christi ;
Liu, Charles ;
Andersen, Richard A. .
SCIENCE, 2015, 348 (6237) :906-910
[2]   Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration [J].
Ajiboye, A. Bolu ;
Willett, Francis R. ;
Young, Daniel R. ;
Memberg, William D. ;
Murphy, Brian A. ;
Miller, Jonathan P. ;
Walter, Benjamin L. ;
Sweet, Jennifer A. ;
Hoyen, Harry A. ;
Keith, Michael W. ;
Peckham, P. Hunter ;
Simeral, John D. ;
Donoghue, John P. ;
Hochberg, Leigh R. ;
Kirsch, Robert F. .
LANCET, 2017, 389 (10081) :1821-1830
[3]   Motor cortex signals for each arm are mixed across hemispheres and neurons yet partitioned within the population response [J].
Ames, Katherine Cora ;
Churchland, Mark M. .
ELIFE, 2019, 8
[4]   Trends and challenges in robot manipulation [J].
Billard, Aude ;
Kragic, Danica .
SCIENCE, 2019, 364 (6446) :1149-+
[5]  
Christie B, 2022, medRxiv, DOI [10.1101/2021.12.28.21268447, 10.1101/2021.12.28.21268447, DOI 10.1101/2021.12.28.21268447]
[6]   Dexterous Control of Seven Functional Hand Movements Using Cortically-Controlled Transcutaneous Muscle Stimulation in a Person With Tetraplegia [J].
Colachis, Samuel C. ;
Bockbrader, Marcie A. ;
Zhang, Mingming ;
Friedenberg, David A. ;
Annetta, Nicholas V. ;
Schwemmer, Michael A. ;
Skomrock, Nicholas D. ;
Mysiw, Walter J. ;
Rezai, Ali R. ;
Bresler, Herbert S. ;
Sharma, Gaurav .
FRONTIERS IN NEUROSCIENCE, 2018, 12
[7]   High-performance neuroprosthetic control by an individual with tetraplegia [J].
Collinger, Jennifer L. ;
Wodlinger, Brian ;
Downey, John E. ;
Wang, Wei ;
Tyler-Kabara, Elizabeth C. ;
Weber, Douglas J. ;
McMorland, Angus J. C. ;
Velliste, Meel ;
Boninger, Michael L. ;
Schwartz, Andrew B. .
LANCET, 2013, 381 (9866) :557-564
[8]   The Motor Cortex Has Independent Representations for Ipsilateral and Contralateral Arm Movements But Correlated Representations for Grasping [J].
Downey, John E. ;
Quick, Kristin M. ;
Schwed, Nathaniel ;
Weiss, Jeffrey M. ;
Wittenberg, George F. ;
Boninger, Michael L. ;
Collinger, Jennifer L. .
CEREBRAL CORTEX, 2020, 30 (10) :5400-5409
[9]   Motor cortical activity changes during neuroprosthetic-controlled object interaction [J].
Downey, John E. ;
Brane, Lucas ;
Gaunt, Robert A. ;
Tyler-Kabara, Elizabeth C. ;
Boninger, Michael L. ;
Collinger, Jennifer L. .
SCIENTIFIC REPORTS, 2017, 7
[10]   Blending of brain-machine interface and vision-guided autonomous robotics improves neuroprosthetic arm performance during grasping [J].
Downey, John E. ;
Weiss, Jeffrey M. ;
Muelling, Katharina ;
Venkatraman, Arun ;
Valois, Jean-Sebastien ;
Hebert, Martial ;
Bagnell, J. Andrew ;
Schwartz, Andrew B. ;
Collinger, Jennifer L. .
JOURNAL OF NEUROENGINEERING AND REHABILITATION, 2016, 13