Pneumatic artificial muscle-driven robot control using local update reinforcement learning

被引:24
|
作者
Cui, Yunduan [1 ]
Matsubara, Takamitsu [1 ]
Sugimoto, Kenji [1 ]
机构
[1] Nara Inst Sci & Technol, Grad Sch Informat Sci, Nara, Japan
关键词
Smooth policy update; dynamic policy programming; robot motor learning; SEARCH;
D O I
10.1080/01691864.2016.1274680
中图分类号
TP24 [机器人技术];
学科分类号
080202 ; 1405 ;
摘要
In this study, a new value function based Reinforcement learning (RL) algorithm, Local Update Dynamic Policy Programming (LUDPP), is proposed. It exploits the nature of smooth policy update using Kullback-Leibler divergence to update its value function locally and considerably reduces the computational complexity. We firstly investigated the learning performance of LUDPP and other algorithms without smooth policy update for tasks of pendulum swing up and n DOFs manipulator reaching in simulation. Only LUDPP could efficiently and stably learn good control policies in high dimensional systems with limited number of training samples. In real word application, we applied LUDPP to control Pneumatic Artificial Muscles (PAMs) driven robots without the knowledge of model which is challenging for traditional methods due to the high nonlinearities of PAM's air pressure dynamics and mechanical structure. LUDPP successfully achieved one finger control of Shadow Dexterous Hand, a PAM-driven humanoid robot hand, with far lower computational resource compared with other conventional value function based RL algorithms.
引用
收藏
页码:397 / 412
页数:16
相关论文
共 50 条
  • [1] Local Update Dynamic Policy Programming in Reinforcement Learning of Pneumatic Artificial Muscle-Driven Humanoid Hand Control
    Cui, Yunduan
    Matsubara, Takamitsu
    Sugimoto, Kenji
    2015 IEEE-RAS 15TH INTERNATIONAL CONFERENCE ON HUMANOID ROBOTS (HUMANOIDS), 2015, : 1083 - 1089
  • [2] Model-Free Adaptive Iterative Learning Control for A Pneumatic Muscle-Driven Robot
    Zhang, Xiong
    Hu, Jiwei
    PROCEEDINGS OF 2019 IEEE 3RD INFORMATION TECHNOLOGY, NETWORKING, ELECTRONIC AND AUTOMATION CONTROL CONFERENCE (ITNEC 2019), 2019, : 2246 - 2249
  • [3] Pneumatic Artificial Muscle-Driven Control Loading System (iFUZZY2017)
    Chen, Chun-Ta
    Wu, Yu-Cheng
    Chen, Fu-Wen
    Chen, Mei-Yung
    INTERNATIONAL JOURNAL OF FUZZY SYSTEMS, 2018, 20 (06) : 1779 - 1789
  • [4] Control System Development for Pneumatic Artificial Muscle-Driven Active Rotor Systems
    Woods, Benjamin K. S.
    Choi, Young-Tai
    Kothera, Curt S.
    Wereley, Norman M.
    JOURNAL OF GUIDANCE CONTROL AND DYNAMICS, 2013, 36 (04) : 1177 - 1185
  • [5] Pneumatic Artificial Muscle-Driven Control Loading System (iFUZZY2017)
    Chun-Ta Chen
    Yu-Cheng Wu
    Fu-Wen Chen
    Mei-Yung Chen
    International Journal of Fuzzy Systems, 2018, 20 : 1779 - 1789
  • [6] A Study of Energy-Efficient and Optimal Locomotion in a Pneumatic Artificial Muscle-Driven Snake Robot
    Lopez, Marcela
    Haghshenas-Jaryani, Mahdi
    ROBOTICS, 2023, 12 (03)
  • [7] A soft artificial muscle driven robot with reinforcement learning
    Tao Yang
    Youhua Xiao
    Zhen Zhang
    Yiming Liang
    Guorui Li
    Mingqi Zhang
    Shijian Li
    Tuck-Whye Wong
    Yong Wang
    Tiefeng Li
    Zhilong Huang
    Scientific Reports, 8
  • [8] A soft artificial muscle driven robot with reinforcement learning
    Yang, Tao
    Xiao, Youhua
    Zhang, Zhen
    Liang, Yiming
    Li, Guorui
    Zhang, Mingqi
    Li, Shijian
    Wong, Tuck-Whye
    Wang, Yong
    Li, Tiefeng
    Huang, Zhilong
    SCIENTIFIC REPORTS, 2018, 8
  • [9] Hysteresis Compensation-Based Intelligent Control for Pneumatic Artificial Muscle-Driven Humanoid Robot Manipulators With Experiments Verification
    Zhang, Xinlin
    Sun, Ning
    Liu, Gendi
    Yang, Tong
    Fang, Yongchun
    IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2024, 21 (03) : 2538 - 2551
  • [10] Hysteresis Compensation-Based Intelligent Control for Pneumatic Artificial Muscle-Driven Humanoid Robot Manipulators With Experiments Verification
    Zhang, Xinlin
    Sun, Ning
    Liu, Gendi
    Yang, Tong
    Fang, Yongchun
    IEEE TRANSACTIONS ON AUTOMATION SCIENCE AND ENGINEERING, 2024, 21 (03) : 2538 - 2551