A Multi-step Neural Control for Motor Brain-Machine Interface by Reinforcement Learning

Brain machine interfaces (BMIs) decode cortical neural spikes of paralyzed patients to control external devices for the purpose of movement restoration. Neuroplasticity induced by conducting a relatively complex task within multi-step is helpful to performance improvements of BMI system. Reinforcement learning (RL) allows the BMI system to interact with the environment to learn the task adaptively without a teacher signal, which is more appropriate to the case for paralyzed patients. In this work, we proposed to apply Q(lambda)-learning to multistep goal-directed tasks using user's neural activity. Neural data were recorded from M1 of a monkey manipulating a joystick in a center-out task. Compared with a supervised learning approach, significant BMI control was achieved with correct directional decoding in 84.2% and 81% of the trials from naive states. The results demonstrate that the BMI system is able to complete a task by interacting with the environment, indicating that RL-based methods have the potential to develop more natural BMI systems.