Reinforcement Learning of Manipulation and Grasping Using Dynamical Movement Primitives for a Humanoidlike Mobile Manipulator

被引:161
作者
Li, Zhijun [1 ]
Zhao, Ting [1 ]
Chen, Fei [2 ]
Hu, Yingbai [1 ]
Su, Chun-Yi [3 ]
Fukuda, Toshio [4 ]
机构
[1] South China Univ Technol, Coll Automat Sci & Engn, Guangzhou 510630, Guangdong, Peoples R China
[2] Ist Italiano Tecnol, Dept Adv Robot, I-16163 Genoa, Italy
[3] Concordia Univ, Dept Mech Ind & Aerosp Engn, Montreal, PQ H4B 1R6, Canada
[4] Beijing Inst Technol, Sch Mechatron Engn, Beijing 100081, Peoples R China
基金
中国国家自然科学基金;
关键词
Dynamic movement primitive (DMP); mobile manipulation; redundancy resolution; reinforcement learning (RL); ROBOT; OPTIMIZATION; FUSION;
D O I
10.1109/TMECH.2017.2717461
中图分类号
TP [自动化技术、计算机技术];
学科分类号
0812 ;
摘要
It is important for humanoid-like mobile robots to learn the complex motion sequences in human-robot environment such that the robots can adapt such motions. This paper describes a reinforcement learning (RL) strategy for manipulation and grasping of a mobile manipulator, which reduces the complexity of the visual feedback and handle varying manipulation dynamics and uncertain external perturbations. Two hierarchies plannings have been considered in the proposed strategy: 1) high-level online redundancy resolution based on the neural-dynamic optimization algorithm in operational space; and 2) low-level RL in joint space. At this level, the dynamic movement primitives have been considered to model and learn the joint trajectories, and then the RL is employed to learn the trajectories with uncertainties. Experimental results on the developed humanoidlike mobile robot demonstrate that the presented approach can suppress the uncertain external perturbations.
引用
收藏
页码:121 / 131
页数:11
相关论文
共 36 条
[1]   Generality and legibility in mobile manipulation [J].
Beetz, Michael ;
Stulp, Freek ;
Esden-Tempski, Piotr ;
Fedrizzi, Andreas ;
Klank, Ulrich ;
Kresse, Ingo ;
Maldonado, Alexis ;
Ruiz, Federico .
AUTONOMOUS ROBOTS, 2010, 28 (01) :21-44
[2]   Visual approach skill for a mobile robot using learning and fusion of simple skills [J].
Boada, MJL ;
Barber, R ;
Salichs, MA .
ROBOTICS AND AUTONOMOUS SYSTEMS, 2002, 38 (3-4) :157-170
[3]   Path Planning for Autonomous Vehicles by Trajectory Smoothing Using Motion Primitives [J].
Bottasso, Carlo L. ;
Leonello, Domenico ;
Savini, Barbara .
IEEE TRANSACTIONS ON CONTROL SYSTEMS TECHNOLOGY, 2008, 16 (06) :1152-1168
[4]   Compliant skills acquisition and multi-optima policy search with EM-based reinforcement learning [J].
Calinon, Sylvain ;
Kormushev, Petar ;
Caldwell, Darwin G. .
ROBOTICS AND AUTONOMOUS SYSTEMS, 2013, 61 (04) :369-379
[5]   AN ONLINE DYNAMIC TRAJECTORY GENERATOR [J].
CASTAIN, RH ;
PAUL, RP .
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH, 1984, 3 (01) :68-72
[6]   THE IMPROVED COMPACT QP METHOD FOR RESOLVING MANIPULATOR REDUNDANCY [J].
CHENG, FT ;
SHEU, RJ ;
CHEN, TH .
IEEE TRANSACTIONS ON SYSTEMS MAN AND CYBERNETICS, 1995, 25 (11) :1521-1530
[7]   Obstacle avoidance of redundant manipulators using neural networks based reinforcement learning [J].
Duguleana, Mihai ;
Barbuceanu, Florin Grigore ;
Teirelbar, Ahmed ;
Mogan, Gheorghe .
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING, 2012, 28 (02) :132-146
[8]   Learning CPG-based biped locomotion with a policy gradient method: Application to a humanoid robot [J].
Endo, Gen ;
Morimoto, Jun ;
Matsubara, Takamitsu ;
Nakanishi, Jun ;
Cheng, Gordon .
INTERNATIONAL JOURNAL OF ROBOTICS RESEARCH, 2008, 27 (02) :213-228
[9]   Haptic Interface for Displaying Softness at Multiple Fingers: Combining a Side-Faced-Type Multifingered Haptic Interface Robot and Improved Softness-Display Devices [J].
Endo, Takahiro ;
Kusakabe, Ayaka ;
Kazama, Yuta ;
Kawasaki, Haruhisa .
IEEE-ASME TRANSACTIONS ON MECHATRONICS, 2016, 21 (05) :2343-2351
[10]   Online optimization scheme with dual-mode controller for redundancy-resolution with torque constraints [J].
Fang, Jian ;
Zhao, Jianghai ;
Mei, Tao ;
Chen, Jian .
ROBOTICS AND COMPUTER-INTEGRATED MANUFACTURING, 2016, 40 :44-54