Human Grasp Feature Learning and Object Recognition Based on Multi-sensor Information Fusion

被引：0

作者：

Zhang Y. ^{[1
]}

Huang Y. ^{[1
,2
]}

Liu Y. ^{[1
]}

Liu C. ^{[1
]}

Liu P. ^{[1
]}

Zhang Y. ^{[1
]}

机构：

[1] School of Electronic Science & Applied Physics, Hefei University of Technology, Hefei

[2] The State Key Laboratory of Bioelectronics, Southeast University, Nanjing

来源：

Jiqiren/Robot | 2020年 / 42卷 / 03期

关键词：

Convolutional neural network; Flexible wearable sensor; Grasp feature; Multi-modal information fusion; Object recognition;

D O I：

10.13973/j.cnki.robot.190353

中图分类号：

学科分类号：

摘要：

Human grasping feature learning and object recognition are studied based on flexible wearable sensors and multi-modal information fusion, and the application of perceptual information to the human grasping process is explored. A data glove is built by utilizing 10 strain sensors, 14 temperature sensors and 78 pressure sensors, and is put on the human hand to measure the bending angle of the finger joints, as well as the temperature and pressure distribution information of the grasped object in human grasping behaviors. The cross-modal information representation is established on time and space sequences, and the multi-modal information is fused by deep convolution neural network to construct the learning model of human grasping feature and realize the accurate recognition of the grasped object. Relevant experiments and validity analysis are carried out for joint angle feature, temperature feature and pressure information feature respectively. The results show that the accurate recognition of 18 kinds of objects can be realized by multi-modal information fusion of multiple sensors. © 2020, Science Press. All right reserved.

引用

页码：267 / 277

页数：10

共 33 条

[21]

Ebrahimzadeh A., Chowdhury M., Maier M., Human-agent-robot task coordination in FiWi-based tactile internet infrastructures using context-and self-awareness, IEEE Transactions on Network and Service Management, 16, 3, pp. 1127-1142, (2019)

[22]

Zhang T., Jiang L., Liu H., Design and functional evaluation of a dexterous myoelectric hand prosthesis with biomimetic tactile sensor, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 26, 7, pp. 1391-1399, (2018)

[23]

Sundaram S., Kellnhofer P., Li Y., Et al., Learning the signatures of the human grasp using a scalable tactile glove, Nature, 569, 7758, (2019)

[24]

Li Y., Zhao C.L., Fei S.J., Et al., E glove rehabilitation evaluation and training system based on ARAT and visual touch fusion, Chinese Journal of Medical Instrumentation, 41, 4, pp. 244-247, (2017)

[25]

Liu H., Qin J., Sun F.C., Et al., Extreme kernel sparse learning for tactile object recognition, IEEE Transactions on Cybernetics, 47, 12, pp. 4509-4520, (2016)

[26]

Kim J.H., Thang N.D., Kim T.S., 3-D hand motion tracking and gesture recognition using a data glove, IEEE International Symposium on Industrial Electronics, pp. 1013-1018, (2009)

[27]

Asif U., Bennamoun M., Sohel F.A., RGB-D object recognition and grasp detection using hierarchical cascaded forests, IEEE Transactions on Robotics, 33, 3, pp. 547-564, (2017)

[28]

Zhang Y.Y., Huang Y., Hao C., Et al., Gestures mapping system based on fabric strain sensor, Chinese Journal of Scientific Instrument, 38, 10, pp. 2422-2429, (2017)

[29]

Zhang Y.Y., Huang Y., Liu J.X., Et al., Sensor design for gesture capturing and master-slave hand motion mapping, Robot, 41, 2, pp. 156-164, (2019)

[30]

Parashar A., Rhu M., Mukkara A., Et al., SCNN: An accelerator for compressed-sparse convolutional neural networks, ACM/IEEE 44th Annual International Symposium on Computer Architecture, pp. 27-40, (2017)

← 1 2 3 4 →