Inertia-Constrained Reinforcement Learning to Enhance Human Motor Control Modeling

被引:5
作者
Korivand, Soroush [1 ,2 ]
Jalili, Nader [1 ]
Gong, Jiaqi [2 ]
机构
[1] Univ Alabama, Dept Mech Engn, Tuscaloosa, AL 35401 USA
[2] Univ Alabama, Dept Comp Sci, Tuscaloosa, AL 35401 USA
关键词
reinforcement learning; locomotion disorder; IMU sensor; musculoskeletal simulation; MUSCLE CONTRIBUTIONS; DYNAMIC SIMULATIONS; OPTIMIZATION; SUPPORT; LEVEL; KNEE; ARM;
D O I
10.3390/s23052698
中图分类号
O65 [分析化学];
学科分类号
070302 ; 081704 ;
摘要
Locomotor impairment is a highly prevalent and significant source of disability and significantly impacts the quality of life of a large portion of the population. Despite decades of research on human locomotion, challenges remain in simulating human movement to study the features of musculoskeletal drivers and clinical conditions. Most recent efforts to utilize reinforcement learning (RL) techniques are promising in the simulation of human locomotion and reveal musculoskeletal drives. However, these simulations often fail to mimic natural human locomotion because most reinforcement strategies have yet to consider any reference data regarding human movement. To address these challenges, in this study, we designed a reward function based on the trajectory optimization rewards (TOR) and bio-inspired rewards, which includes the rewards obtained from reference motion data captured by a single Inertial Moment Unit (IMU) sensor. The sensor was equipped on the participants' pelvis to capture reference motion data. We also adapted the reward function by leveraging previous research on walking simulations for TOR. The experimental results showed that the simulated agents with the modified reward function performed better in mimicking the collected IMU data from participants, which means that the simulated human locomotion was more realistic. As a bio-inspired defined cost, IMU data enhanced the agent's capacity to converge during the training process. As a result, the models' convergence was faster than those developed without reference motion data. Consequently, human locomotion can be simulated more quickly and in a broader range of environments, with a better simulation performance.
引用
收藏
页数:16
相关论文
共 68 条
  • [1] Optimality principles for model-based prediction of human gait
    Ackermann, Marko
    van den Bogert, Antonie J.
    [J]. JOURNAL OF BIOMECHANICS, 2010, 43 (06) : 1055 - 1060
  • [2] Akhavan Z., 2022, ARXIV
  • [3] Akimov D., 2019, ARXIV
  • [4] Dynamic optimization of human walking
    Anderson, FC
    Pandy, MG
    [J]. JOURNAL OF BIOMECHANICAL ENGINEERING-TRANSACTIONS OF THE ASME, 2001, 123 (05): : 381 - 390
  • [5] A consecutive hybrid spiking-convolutional (CHSC) neural controller for sequential decision making in robots
    Azimirad, Vahid
    Ramezanlou, Mohammad Tayefe
    Sotubadi, Saleh Valizadeh
    Janabi-Sharifi, Farrokh
    [J]. NEUROCOMPUTING, 2022, 490 : 319 - 336
  • [6] Spectrum-Aware Mobile Edge Computing for UAVs Using Reinforcement Learning
    Badnava, Babak
    Kim, Taejoon
    Cheung, Kenny
    Ali, Zaheer
    Hashemi, Morteza
    [J]. 2021 ACM/IEEE 6TH SYMPOSIUM ON EDGE COMPUTING (SEC 2021), 2021, : 376 - 380
  • [7] Subsensory electrical noise stimulation applied to the lower trunk improves postural control during visual perturbations
    Bassiri, Zahra
    Austin, Caroline
    Cousin, Christian
    Martelli, Dario
    [J]. GAIT & POSTURE, 2022, 96 : 22 - 28
  • [8] Machine Learning Algorithms Can Use Wearable Sensor Data to Accurately Predict Six-Week Patient-Reported Outcome Scores Following Joint Replacement in a Prospective Trial
    Bini, Stefano A.
    Shah, Romil E.
    Bendich, Ilya
    Patterson, Joseph T.
    Hwang, Kevin M.
    Zaid, Musa B.
    [J]. JOURNAL OF ARTHROPLASTY, 2019, 34 (10) : 2242 - 2247
  • [9] Real-time myoprocessors for a neural controlled powered exoskeleton arm
    Cavallaro, Ettore E.
    Rosen, Jacob
    Perry, Joel C.
    Burns, Stephen
    [J]. IEEE TRANSACTIONS ON BIOMEDICAL ENGINEERING, 2006, 53 (11) : 2387 - 2396
  • [10] Chandler R.F., 1975, INVESTIGATION INERTI