Multi-sensor fusion based industrial action recognition method under the environment of intelligent manufacturing

被引：7

作者：

Wang Z. ^{[1
]}

Yan J. ^{[1
]}

机构：

[1] School of Mechatronics Engineering, Harbin Institute of Technology, Harbin

来源：

Journal of Manufacturing Systems | 2024年 / 74卷

关键词：

Action recognition; D-S evidence theory; Industry; 4.0; Intelligent manufacturing; Multi-sensor fusion;

D O I：

10.1016/j.jmsy.2024.04.019

中图分类号：

学科分类号：

摘要：

In the context of intelligent manufacturing and Industry 4.0, the manufacturing industry is rapidly transitioning toward mass personalization production. Despite this trend, the assembly industry still relies on manual operations performed by workers, considering their cognitive ability and flexibility. Thereinto, studying operator action perception and recognition methods is a vital filed and of great significance for improving the production efficiency and ensuring product quality. In this paper, a multi-sensor fusion-based data acquisition system is constructed to address the challenge of achieving comprehensive and accurate perception of the assembly process with a single sensor. Then, an action recognition model architecture based on ResNet + LSTM + D-S evidence theory is proposed and established. By fully considering the characteristics of different data, the multi-sensor data values are maximized, data complementarity is achieved, and the recognition accuracy exceeds 97 %. This research is expected to provide guidance for increasing the degree of workshop automation and improving the efficiency and quality of the production process. © 2024

引用

页码：575 / 586

页数：11

共 64 条

[21]

Wang L., Li W., Li W., Et al., Appearance-and-relation networks for video classification, In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1430-1439, (2018)

[22]

Zhang J.H., Lan C.L., Xing J.L., Et al., View adaptive neural networks for high performance skeleton-based human action recognition, IEEE Trans Pattern Anal Mach Intell, pp. 1963-1978, (2019)

[23]

Yan S.J., Xiong Y.J., Lin D.H., Spatial temporal graph convolutional networks for skeleton-based action recognition, AIn: proceedings of the thirty second AAAI conference on artificial intelligencel, pp. 7444-7452, (2018)

[24]

Plizzari C., Cannici M., Matteucci M., Spatial temporal transformer network for skeleton-based action recognition, Int Conf Pattern Recognit, pp. 694-701, (2021)

[25]

Zhang J., Wang P., Gao R.X., Hybrid machine learning for human action recognition and prediction in assembly, Robot Comput-Integr Manuf, 72, (2021)

[26]

Zhang J., HongyiChang Q., LihuiGao, Robert X., Recurrent neural network for motion trajectory prediction in human-robot collaborative assembly, CIRP Ann, 69, (2020)

[27]

Liu T., Weng C., Jiao L., Huang J., Wang X., Ni Z., Et al., Toward fast 3D human activity recognition: a refined feature based on minimum joint freedom model (Mint), J Manuf Syst, 66, pp. 127-141, (2023)

[28]

Wu H.B., Xin M., Yi B.L., Et al., Spatiotemporal multimodal learning with 3D CNNs for video action recognition, IEEE Trans Circuits Syst Video Technol, (2021)

[29]

Donahue J., Hendricks L.A., Rohrbach M., Et al., Long-term recurrent convolutional networks for visual recognition and description, IEEE Conf Comput Vis Pattern Recognit, pp. 677-691, (2015)

[30]

Xiong Q., Zhang J., Wang P., Liu D., Gao R.X., Transferable two-stream convolutional neural network for human action recognition, J Manuf Syst, 56, pp. 605-614, (2020)

← 1 2 3 4 5 6 7 →