Fast Micro-Expression recognition method based on Bi-Directional optical flow

被引:0
作者
Zhang, Yukun [1 ]
Fei, Zixiang [2 ]
Zhou, Wenju [1 ]
Fei, Minrui [1 ]
机构
[1] Shanghai Univ, Sch Mechatron Engn & Automat, Shanghai 200444, Peoples R China
[2] Shanghai Univ, Sch Comp Engn & Sci, Shanghai 200444, Peoples R China
关键词
Micro-expression; Bi-directional optical flow; Keyframe; Emotion recognition; Facial movement;
D O I
10.1007/s10489-025-06722-9
中图分类号
TP18 [人工智能理论];
学科分类号
081104 ; 0812 ; 0835 ; 1405 ;
摘要
Micro-expressions contain rich emotional and cognitive information, and can serve as important indicators of mental status. Systematic observation and precise recognition of facial micro-expressions in the elderly can provide critical and valuable clues for early diagnosis of dementia, dynamic monitoring of the disease, and evaluation of treatment effects. However, the micro-expression motion intensity is weak, and the traditional optical flow-based micro-expression recognition method is prone to extracting erroneous optical flow information, which affects the recognition accuracy of the system. This paper proposes a Bi-directional Optical flow-based Fast Micro-expression recognition method (BOFM), which employs forward and reverse bi-directional optical flow to capture facial movements and accurately recognize micro-expressions. Additionally, it introduces a keyframe extraction method that utilizes the variation effect of the optical flow field to eliminate unnecessary frames in video clips and enhance the real-time performance of the system. This method has been validated using public datasets such as SMIC and CASME. The verification results demonstrate that this method achieves approximately a 9.3% higher accuracy compared to the Sparse MDMO (Main Directional Mean Optical-flow) algorithm. Notably, the proposed algorithm showcases a significant running time reduction of approximately 36.5% when compared to the micro-expression recognition algorithm based on FlowNet2. These findings clearly indicate that the proposed algorithm possesses excellent capabilities in recognizing micro-expressions. This establishes a basis for future investigations into the connection between facial micro-expression patterns and early-stage Alzheimer's disease, playing a crucial role in its early detection and prevention.
引用
收藏
页数:20
相关论文
共 64 条
[1]   Self-Supervised Approach for Facial Movement Based Optical Flow [J].
Alkaddour, Muhannad ;
Tariq, Usman ;
Dhall, Abhinav .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2022, 13 (04) :2071-2085
[2]  
Ameur B, 2016, 2016 2ND INTERNATIONAL CONFERENCE ON ADVANCED TECHNOLOGIES FOR SIGNAL AND IMAGE PROCESSING (ATSIP), P453, DOI 10.1109/ATSIP.2016.7523134
[3]   An Optical Flow Based Approach for Facial Expression Recognition [J].
Anthwal, Shivangi ;
Ganotra, Dinesh .
2019 INTERNATIONAL CONFERENCE ON POWER ELECTRONICS, CONTROL AND AUTOMATION (ICPECA-2019), 2019, :387-391
[4]   Boosting Micro-Expression Recognition via Self-Expression Reconstruction and Memory Contrastive Learning [J].
Bao, Yongtang ;
Wu, Chenxi ;
Zhang, Peng ;
Shan, Caifeng ;
Qi, Yue ;
Ben, Xianye .
IEEE TRANSACTIONS ON AFFECTIVE COMPUTING, 2024, 15 (04) :2083-2096
[5]   SLIM: Self-Supervised LiDAR Scene Flow and Motion Segmentation [J].
Baur, Stefan Andreas ;
Emmerichs, David Josef ;
Moosmann, Frank ;
Pinggera, Peter ;
Ommer, Bjoern ;
Geiger, Andreas .
2021 IEEE/CVF INTERNATIONAL CONFERENCE ON COMPUTER VISION (ICCV 2021), 2021, :13106-13116
[6]   Video-Based Facial Micro-Expression Analysis: A Survey of Datasets, Features and Algorithms [J].
Ben, Xianye ;
Ren, Yi ;
Zhang, Junping ;
Wang, Su-Jing ;
Kpalma, Kidiyo ;
Meng, Weixiao ;
Liu, Yong-Jin .
IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2022, 44 (09) :5826-5846
[7]   Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods [J].
Bruhn A. ;
Weickert J. ;
Schnörr C. .
International Journal of Computer Vision, 2005, 61 (3) :1-21
[8]   A Naturalistic Open Source Movie for Optical Flow Evaluation [J].
Butler, Daniel J. ;
Wulff, Jonas ;
Stanley, Garrett B. ;
Black, Michael J. .
COMPUTER VISION - ECCV 2012, PT VI, 2012, 7577 :611-625
[9]   LIBSVM: A Library for Support Vector Machines [J].
Chang, Chih-Chung ;
Lin, Chih-Jen .
ACM TRANSACTIONS ON INTELLIGENT SYSTEMS AND TECHNOLOGY, 2011, 2 (03)
[10]   Block Division Convolutional Network With Implicit Deep Features Augmentation for Micro-Expression Recognition [J].
Chen, Bin ;
Liu, Kun-Hong ;
Xu, Yong ;
Wu, Qing-Qiang ;
Yao, Jun-Feng .
IEEE TRANSACTIONS ON MULTIMEDIA, 2023, 25 :1345-1358