A Multi-Module Sensing and Bi-Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

被引:13
作者
Fang, Ping [1 ]
Zhu, Ming-lu [1 ]
Zeng, Zhang-bing [1 ]
Lu, Wen-xiao [1 ]
Wang, Feng-Xia [1 ]
Zhang, Lu [1 ]
Chen, Tao [2 ]
Sun, Li-ning [1 ]
机构
[1] Soochow Univ, Sch Mech & Elect Engn, Prov Key Lab Adv Robot, Suzhou 215123, Peoples R China
[2] Soochow Univ, Sch Future Sci & Engn, Suzhou 215299, Peoples R China
基金
中国国家自然科学基金;
关键词
bi-directional interaction; human-machine interface; intelligent robot; machine learning; multi-modal sensing; ELECTROMYOGRAPHY; TIME;
D O I
10.1002/adfm.202310254
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Rapid advances in robotics have placed urgent demands on more intelligent human-machine interaction technologies. Specifically, the way of establishing dual-way intuitive communication with a consistent sensory system can greatly enhance efficiency and reliability. Here, a bi-directional human-machine interface (HMI) is designed by applying starch-based hydrogel sensors. The whole system consists of a multi-modal wearable sensory exoskeleton with a haptic feedback module and sensory robotic hand. The sensory exoskeleton with strain-sensing glove and rotation-sensing arm can capture and project the motion of the entire upper limb. The system offers object recognition functions by utilizing a sensing array on the robotic hands and machine learning algorithms, which can identify the shape and hardness information. The recognized results can be delivered back to the operator via vision and vibrational haptic feedback, respectively. This dual-way intelligent sensory system shows potential application in many key fields such as the Internet of Things, teleoperation, and medical robotics. A bi-directional interaction system with a human-machine interaction unit and a machine-human feedback unit is proposed. This system combines multimodal sensing, mechanical structures, intelligent algorithms, and recognition vibrations. Importantly, this system is able to provide more information for new human-machine interaction interfaces, showing the positive prospects of intelligent robot teleoperation.image
引用
收藏
页数:15
相关论文
共 89 条
[61]   Self-Powered Gesture Recognition Wristband Enabled by Machine Learning for Full Keyboard and Multicommand Input [J].
Tan, Puchuan ;
Han, Xi ;
Zou, Yang ;
Qu, Xuecheng ;
Xue, Jiangtao ;
Li, Tong ;
Wang, Yiqian ;
Luo, Ruizeng ;
Cui, Xi ;
Xi, Yuan ;
Wu, Le ;
Xue, Bo ;
Luo, Dan ;
Fan, Yubo ;
Chen, Xun ;
Li, Zhou ;
Wang, Zhong Lin .
ADVANCED MATERIALS, 2022, 34 (21)
[62]   Design, manufacturing and applications of wearable triboelectric nanogenerators [J].
Wang, Haobin ;
Han, Mengdi ;
Song, Yu ;
Zhang, Haixia .
NANO ENERGY, 2021, 81
[63]   Stretchable and Shape-Adaptable Triboelectric Nanogenerator Based on Biocompatible Liquid Electrolyte for Biomechanical Energy Harvesting and Wearable Human-Machine Interaction [J].
Wang, Liangliang ;
Liu, Wenquan ;
Yan, Zhengguang ;
Wang, Feijiu ;
Wang, Xin .
ADVANCED FUNCTIONAL MATERIALS, 2021, 31 (07)
[64]   Self-Powered High-Resolution and Pressure-Sensitive Triboelectric Sensor Matrix for Real-Time Tactile Mapping [J].
Wang, Xiandi ;
Zhang, Hanlu ;
Dong, Lin ;
Han, Xun ;
Du, Weiming ;
Zhai, Junyi ;
Pan, Caofeng ;
Wang, Zhong Lin .
ADVANCED MATERIALS, 2016, 28 (15) :2896-2903
[65]   Naturally sourced hydrogels: emerging fundamental materials for next-generation healthcare sensing [J].
Wang, Zhenwu ;
Wei, Hua ;
Huang, Youju ;
Wei, Yen ;
Chen, Jing .
CHEMICAL SOCIETY REVIEWS, 2023, 52 (09) :2992-3034
[66]   Stretchable freezing-tolerant triboelectric nanogenerator and strain sensor based on transparent, long-term stable, and highly conductive gelatin-based organohydrogel [J].
Wu, Min ;
Wang, Xin ;
Xia, Yifan ;
Zhu, Yan ;
Zhu, Shunli ;
Jia, Chunyang ;
Guo, Wenyu ;
Li, Qingqing ;
Yan, Zhengguang .
NANO ENERGY, 2022, 95
[67]   Boosting output performance of triboelectric nanogenerator based on BaTiO3:La embedded nanofiber membrane for energy harvesting and wireless power transmission [J].
Xi, Benbo ;
Wang, Liangliang ;
Yang, Biao ;
Xia, Yifan ;
Chen, Delu ;
Wang, Xin .
NANO ENERGY, 2023, 110
[68]   Flexible Hybrid Sensor Systems with Feedback Functions [J].
Xu, Kaichen ;
Lu, Yuyao ;
Takei, Kuniharu .
ADVANCED FUNCTIONAL MATERIALS, 2021, 31 (39)
[69]   Flexible High-Resolution Triboelectric Sensor Array Based on Patterned Laser-Induced Graphene for Self-Powered Real-Time Tactile Sensing [J].
Yan, Zhengguang ;
Wang, Liangliang ;
Xia, Yifan ;
Qiu, Rendong ;
Liu, Wenquan ;
Wu, Min ;
Zhu, Yan ;
Zhu, Shunli ;
Jia, Chunyang ;
Zhu, Miaomiao ;
Cao, Ruirui ;
Li, Zhaoling ;
Wang, Xin .
ADVANCED FUNCTIONAL MATERIALS, 2021, 31 (23)
[70]   Self-Healing Hydrogels: From Synthesis to Multiple Applications [J].
Yin, Hongyan ;
Liu, Fangfei ;
Abdiryim, Tursun ;
Liu, Xiong .
ACS MATERIALS LETTERS, 2023, 5 (07) :1787-1830