Volume 42 Issue 7
Aug 2022
Turn off MathJax
Article Contents
JOURNAL OF MECHANICAL ENGINEERING  > 2022  >  42(7): 749-754.
ZHENG Shuhua, CHEN Mengxin, WANG Xiangzhou, GONG Xueya. A Micro-Expression Apex Frame Spotting Method Based on Optical-Flow-Dual-Input Network[J]. JOURNAL OF MECHANICAL ENGINEERING, 2022, 42(7): 749-754. doi: 10.15918/j.tbit1001-0645.2021.135
Citation: ZHENG Shuhua, CHEN Mengxin, WANG Xiangzhou, GONG Xueya. A Micro-Expression Apex Frame Spotting Method Based on Optical-Flow-Dual-Input Network[J]. JOURNAL OF MECHANICAL ENGINEERING, 2022, 42(7): 749-754. doi: 10.15918/j.tbit1001-0645.2021.135
shu

A Micro-Expression Apex Frame Spotting Method Based on Optical-Flow-Dual-Input Network

doi: 10.15918/j.tbit1001-0645.2021.135

  • School of Automation, Beijing Institute of Technology, Beijing 100081, China

  • Received Date: 17 May 2021
  • Accepted Date: 17 May 2021
  • Issue Publish Date: 17 Aug 2022
    Abstract
  • Micro-expression apex frame contains abundant micro-expression information. In order to spot the apex frame accurately, a neural network classification was proposed based on optical flow characteristics. Taking prior knowledge as rules, a detection method was designed to realize micro-expression apex frame spotting. Firstly, optical flow information was extracted from the image in a fixed size sliding window. And then, the spatial and temporal features of optical flow information in x and y directions was extracted and classified based on dual input network. Finally, according to the trade-off rules based on prior knowledge of micro expression, a post-processing was carried out to improve the detection accuracy. The experimental results on data set CASMEⅡtesting show that the apex spotting rate (ASR) and F1-score can reach up to 0.945 and 0.925 respectively.

     

    • FullText(HTML)
  • loading
    • References(19)
  • [1]
    解仑, 卢亚楠, 姜波, 等. 基于人脸运动单元及表情关系模型的自动表情识别[J]. 北京理工大学学报, 2016, 36(2):163 − 169.

    XIE Lun, LU Yanan, JIANG Bo, et al. Expression automatic recognition based on facial action units and expression relationship model[J]. Transactions of Beijing Institute of Technology, 2016, 36(2):163 − 169. (in Chinese)
    [2]
    岳雷, 沈庭芝. 基于自动提取特征点的三维人脸表情识别[J]. 北京理工大学学报, 2016, 36(5):508 − 513.

    YUE Lei, SHEN Tingzhi. 3D expression recognition based on automatically detected facial points[J]. Transactions of Beijing Institute of Technology, 2016, 36(5):508 − 513. (in Chinese)
    [3]
    LE NGO A C, PHAN R C W, SEE J. Spontaneous subtle expression recognition: imbalanced databases and solutions[C]//Proceedings of Asian Conference on Computer Vision. Cham: Springer, 2014: 33-48.
    [4]
    LI X, HONG X, MOILANEN A, et al. Towards reading hidden emotions: a comparative study of spontaneous micro-expression spotting and recognition methods[J]. IEEE Transactions on Affective Computing, 2017, 9(4):563 − 577.
    [5]
    DAVISON A, MERGHANI W, LANSLEY C, et al. Objective micro-facial movement detection using facs-based regions and baseline evaluation[C]//Proceedings of 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). [S. l. ]: IEEE, 2018: 642 − 649.
    [6]
    DING J, TIAN Z, LYU X, et al. Real-time micro-expression detection in unlabeled long videos using optical flow and lstm neural network[C]//Proceedings of International Conference on Computer Analysis of Images and Patterns. [S. l. ]: Springer, 2019: 622 − 634.
    [7]
    TAKALKAR M A, THUSEETHAN S, RAJASEGARAR S, et al. LGAttNet: automatic micro-expression detection using dual-stream local and global attentions[J]. Knowledge-Based Systems, 2021, 212:106566. doi: 10.1016/j.knosys.2020.106566
    [8]
    ESMAEILI V, FEGHHI M M, SHAHDI S O. Automatic micro-expression apex frame spotting using local binary pattern from six intersection planes[J/OL]. [2021-04-05]. https://arxiv.org/abs/2104.02149.
    [9]
    TRAN T K, VO Q N, HONG X, et al. Dense prediction for micro-expression spotting based on deep sequence model[J]. Electronic Imaging, 2019, 2019(8):401−1 − 401−6. doi: 10.2352/ISSN.2470-1173.2019.8.IMAWM-401
    [10]
    ZHANG Z, CHEN T, MENG H, et al. SMEConvNet: a convolutional neural network for spotting spontaneous facial micro-expression from long videos[J]. IEEE Access, 2018, 6:71143 − 71151. doi: 10.1109/ACCESS.2018.2879485
    [11]
    DONAHUE J, HENDRICKS A L, GUADARRAMA S, et al. Long-term recurrent convolutional networks for visual recognition and description[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. [S. l. ]: IEEE, 2015: 2625 − 2634.
    [12]
    SUN B, CAO S, HE J, et al. Two-stream attention-aware network for spontaneous micro-expression movement spotting[C]//Proceedings of 2019 IEEE 10th International Conference on Software Engineering and Service Science (ICSESS). [S. l. ]: IEEE, 2019: 702 − 705.
    [13]
    ZACH C, POCK T, BISCHOF H. A duality based approach for realtime tv-l 1 optical flow[C]//Proceedings of Joint Pattern Recognition Symposium. Berlin, Heidelberg: Springer, 2007: 214 − 223.
    [14]
    XIA B, WANG W, WANG S, et al. Learning from macro-expression: a micro-expression recognition framework[C]//Proceedings of the 28th ACM International Conference on Multimedia. [S. l. ]: ACM, 2020: 2936 − 2944.
    [15]
    YAN W J, LI X, WANG S J, et al. CASMEⅡ: an improved spontaneous micro-expression database and the baseline evaluation[J]. PloS one, 2014, 9(1):e86041. doi: 10.1371/journal.pone.0086041
    [16]
    PAN H, XIE L, WANG Z, et al. Review of micro-expression spotting and recognition in video sequences[J]. Virtual Reality & Intelligent Hardware, 2021, 3(1):1 − 17.
    [17]
    OH Y H, SEE J, LE NGO A C, et al. A survey of automatic facial micro-expression analysis: databases, methods, and challenges[J]. Frontiers in Psychology, 2018, 9:1128. doi: 10.3389/fpsyg.2018.01128
    [18]
    TRAN T K, VO Q N, HONG X, et al. Dense prediction for micro-expression spotting based on deep sequence model[J]. Electronic Imaging, 2019(8):401 − 406.
    [19]
    FU X, NIU L, HU Z, et al. Deep micro-expression spotting network training based on concept of transition frame[J]. Journal of ZheJiang University (Engineering Science), 2020, 54(11):2128 − 2137.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    Figures(5)  / Tables(4)

    Get Citation
    PDF
    XML

    Article Metrics

    Article views(82) PDF downloads(0) Cited by()
    Proportional views
    Related

    Copyright © 2009《 石油钻探技术 》 All Rights Reserved.

    Address：North-Star Times Tower, No.8 Beichendong Road, Chaoyang District, Beijing, China China Pos：100101

    Tel：010-84988317，84988356，84988357 Fax：021-64253812 Email：syzt@vip.163.com

    Beijing public network security equipment 11010502036328　Beijing ICP backup 17033152

    Supported by: Beijing Renhe Information Technology Co., Ltd.  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return