A Multi-Module Sensing and Bi-Directional HMI Integrating Interaction, Recognition, and Feedback for Intelligent Robots

被引:13
作者
Fang, Ping [1 ]
Zhu, Ming-lu [1 ]
Zeng, Zhang-bing [1 ]
Lu, Wen-xiao [1 ]
Wang, Feng-Xia [1 ]
Zhang, Lu [1 ]
Chen, Tao [2 ]
Sun, Li-ning [1 ]
机构
[1] Soochow Univ, Sch Mech & Elect Engn, Prov Key Lab Adv Robot, Suzhou 215123, Peoples R China
[2] Soochow Univ, Sch Future Sci & Engn, Suzhou 215299, Peoples R China
基金
中国国家自然科学基金;
关键词
bi-directional interaction; human-machine interface; intelligent robot; machine learning; multi-modal sensing; ELECTROMYOGRAPHY; TIME;
D O I
10.1002/adfm.202310254
中图分类号
O6 [化学];
学科分类号
0703 ;
摘要
Rapid advances in robotics have placed urgent demands on more intelligent human-machine interaction technologies. Specifically, the way of establishing dual-way intuitive communication with a consistent sensory system can greatly enhance efficiency and reliability. Here, a bi-directional human-machine interface (HMI) is designed by applying starch-based hydrogel sensors. The whole system consists of a multi-modal wearable sensory exoskeleton with a haptic feedback module and sensory robotic hand. The sensory exoskeleton with strain-sensing glove and rotation-sensing arm can capture and project the motion of the entire upper limb. The system offers object recognition functions by utilizing a sensing array on the robotic hands and machine learning algorithms, which can identify the shape and hardness information. The recognized results can be delivered back to the operator via vision and vibrational haptic feedback, respectively. This dual-way intelligent sensory system shows potential application in many key fields such as the Internet of Things, teleoperation, and medical robotics. A bi-directional interaction system with a human-machine interaction unit and a machine-human feedback unit is proposed. This system combines multimodal sensing, mechanical structures, intelligent algorithms, and recognition vibrations. Importantly, this system is able to provide more information for new human-machine interaction interfaces, showing the positive prospects of intelligent robot teleoperation.image
引用
收藏
页数:15
相关论文
共 89 条
[41]   Smartphone recognition-based immune microparticles for rapid on-site visual data-sharing detection of Newcastle disease virus [J].
Lu, Shenjunjie ;
Ma, Binglin ;
Zhou, Haoran ;
Li, Yayang ;
Qiao, Zhihao ;
Xiao, Tong ;
Li, Husheng ;
Wang, Baijie ;
Cui, Meihui ;
Zhang, Shufang ;
Chang, Jin ;
Du, Taofeng ;
Liu, Jing ;
Wang, Hanjie .
TALANTA, 2023, 252
[42]   Multifunctional Flexible Humidity Sensor Systems Towards Noncontact Wearable Electronics [J].
Lu, Yuyao ;
Yang, Geng ;
Shen, Yajing ;
Yang, Huayong ;
Xu, Kaichen .
NANO-MICRO LETTERS, 2022, 14 (01)
[43]   Functionalized Hydrogel-Based Wearable Gas and Humidity Sensors [J].
Luo, Yibing ;
Li, Jianye ;
Ding, Qiongling ;
Wang, Hao ;
Liu, Chuan ;
Wu, Jin .
NANO-MICRO LETTERS, 2023, 15 (01)
[44]   Self-powered multifunctional body motion detectors based on highly compressible and stretchable ferroelectrets with an air-filled parallel-tunnel structure [J].
Ma, Xingchen ;
Chen, Xin ;
Xiang, Xinhao ;
Zhang, Fei ;
Zhao, Yanjun ;
Wang, Fayang ;
Mu, Xiaojing ;
Dai, Ying ;
He, Pengfei ;
Zhang, Xiaoqing .
NANO ENERGY, 2022, 103
[45]   Wearables for Running Gait Analysis: A Systematic Review [J].
Mason, Rachel ;
Pearson, Liam T. ;
Barry, Gillian ;
Young, Fraser ;
Lennon, Oisin ;
Godfrey, Alan ;
Stuart, Samuel .
SPORTS MEDICINE, 2023, 53 (01) :241-268
[46]   Hybrid-Flexible Bimodal Sensing Wearable Glove System for Complex Hand Gesture Recognition [J].
Pan, Jieming ;
Li, Yida ;
Luo, Yuxuan ;
Zhang, Xiangyu ;
Wang, Xinghua ;
Wong, David Liang Tai ;
Heng, Chun-Huat ;
Tham, Chen-Khong ;
Thean, Aaron Voon-Yew .
ACS SENSORS, 2021, 6 (11) :4156-4166
[47]   Frequency-selective acoustic and haptic smart skin for dual-mode dynamic/static human-machine interface [J].
Park, Jonghwa ;
Kang, Dong-Hee ;
Chae, Heeyoung ;
Ghosh, Sujoy Kumar ;
Jeong, Changyoon ;
Park, Yoojeong ;
Cho, Seungse ;
Lee, Youngoh ;
Kim, Jinyoung ;
Ko, Yujung ;
Kim, Jae Joon ;
Ko, Hyunhyub .
SCIENCE ADVANCES, 2022, 8 (12)
[48]   Individually addressable and flexible pressure sensor matrixes with ZnO nanotube arrays on graphene [J].
Park, Junbeom ;
Ghosh, Ramesh ;
Song, Minho S. ;
Hwang, Yunjae ;
Tchoe, Youngbin ;
Saroj, Rajendra Kumar ;
Ali, Asad ;
Guha, Puspendu ;
Kim, Bosung ;
Kim, Sang-Woo ;
Kim, Miyoung ;
Yi, Gyu-Chul .
NPG ASIA MATERIALS, 2022, 14 (01)
[49]   Recent Progress in Flexible Tactile Sensors for Human-Interactive Systems: From Sensors to Advanced Applications [J].
Pyo, Soonjae ;
Lee, Jaeyong ;
Bae, Kyubin ;
Sim, Sangjun ;
Kim, Jongbaeg .
ADVANCED MATERIALS, 2021, 33 (47)
[50]   Bioinspired, multifunctional dual -mode pressure sensors as electronic skin for decoding complex loading processes and human motions [J].
Qiu, Ye ;
Tian, Ye ;
Sun, Shenshen ;
Hu, Jiahui ;
Wang, Youyan ;
Zhang, Zheng ;
Liu, Aiping ;
Cheng, Huanyu ;
Gao, Weizhan ;
Zhang, Wenan ;
Chai, Hao ;
Wu, Huaping .
NANO ENERGY, 2020, 78 (78)