Robust real-time hand detection and localization for space human-robot interaction based on deep learning

被引:84
作者
Gao, Qing [1 ,2 ]
Liu, Jinguo [1 ]
Ju, Zhaojie [1 ,3 ]
机构
[1] Chinese Acad Sci, Shenyang Inst Automat, Inst Robot & Intelligent Mfg, State Key Lab Robot, Shenyang, Peoples R China
[2] Univ Chinese Acad Sci, Beijing, Peoples R China
[3] Univ Portsmouth, Sch Comp, Portsmouth, Hants, England
基金
国家重点研发计划; 中国国家自然科学基金;
关键词
Astronaut assistant robot; Deep learning; Hand detection and localization; SSD;
D O I
10.1016/j.neucom.2019.02.066
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Hand gestures are quite suitable for space human-robot interaction (SHRI) because of their natural and convenient features. While the detection and localization of hands are the premise and foundation for SHRI based on hand gestures. But hand gestures are very complicated and hand sizes are very small in some images. These problems make the robust real-time hand detection and localization very difficult. In this paper, a feature-map-fused single shot multibox detector (FF-SSD) which is a deep learning network is designed to deal with the problems of hand detection and localization in SHRI. First, the background of the method is introduced in this paper, including an astronaut assistant robot platform, the difficulties of hand detection and localization, and introduction of the state-of-the-art deep learning networks for object detection and localization. Then, the FF-SSD is proposed for detecting and localizing hands especially pony-size hands. This network takes into consideration both accuracy and speed with balanced performance. And in the experiment part, the FF-SSD is trained and tested on hand databases which include a homemade database and two public databases. At last, the superiority of the proposed method is demonstrated compared with the state-of-the-art methods. (C) 2019 Elsevier B.V. All rights reserved.
引用
收藏
页码:198 / 206
页数:9
相关论文
共 28 条
[1]  
[Anonymous], HELLO I AM CIMON
[2]  
[Anonymous], 2017, COMPUT RES REPOS
[3]  
[Anonymous], BRIT MACH VIS C BMVC
[4]  
[Anonymous], SMART SPHERES TELERO
[5]   Lending A Hand: Detecting Hands and Recognizing Activities in Complex Egocentric Interactions [J].
Bambach, Sven ;
Lee, Stefan ;
Crandall, David J. ;
Yu, Chen .
2015 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV), 2015, :1949-1957
[6]  
Bualat M, 2015, AIAA SPACE 2015 Conference and Exposition, P4643, DOI DOI 10.2514/6.2015-4643
[7]   Feature-Fused SSD: Fast Detection for Small Objects [J].
Cao, Guimei ;
Xie, Xuemei ;
Yang, Wenzhe ;
Liao, Quan ;
Shi, Guangming ;
Wu, Jinjian .
NINTH INTERNATIONAL CONFERENCE ON GRAPHIC AND IMAGE PROCESSING (ICGIP 2017), 2018, 10615
[8]  
Dorais G. A., 2003, 2003 IEEE Aerospace Conference Proceedings (Cat. No.03TH8652), P1
[9]   Static Hand Gesture Recognition with Parallel CNNs for Space Human-Robot Interaction [J].
Gao, Qing ;
Liu, Jinguo ;
Ju, Zhaojie ;
Li, Yangmin ;
Zhang, Tian ;
Zhang, Lu .
INTELLIGENT ROBOTICS AND APPLICATIONS, ICIRA 2017, PT I, 2017, 10462 :462-473
[10]   Free-flying dynamics and control of an astronaut assistant robot based on fuzzy sliding mode algorithm [J].
Gao, Qing ;
Liu, Jinguo ;
Tian, Tongtong ;
Li, Yangmin .
ACTA ASTRONAUTICA, 2017, 138 :462-474