Dynamic Expression Recognition Based on Dynamic Time Warping and Active Appearance Model
XU Liangfeng① WANG Jiayong① CUI Jingnan② HU Min① ZHANG Keke① TENG Wendi①
①(School of Computer and Information, Hefei University of Technology, Hefei 230009, China) ②(School of Computer and Information, South China University of Technology, Guangzhou 510006, China)
摘要 针对静态表情特征缺乏时间信息，不能充分体现表情的细微变化，该文提出一种针对非特定人的动态表情识别方法：基于动态时间规整(Dynamic Time Warping, DTW)和主动外观模型(Active Appearance Model, AAM)的动态表情识别。首先采用基于局部梯度DT-CWT(Dual-Tree Complex Wavelet Transform)主方向模式(Dominant Direction Pattern, DDP)特征的DTW对表情序列进行规整。然后采用AAM定位出表情图像的66个特征点并进行跟踪，利用中性脸的特征点构建人脸几何模型，通过人脸几何模型的匹配克服不同人呈现表情的差异，并通过计算表情序列中相邻两帧图像对应特征点的位移获得表情的变化特征。最后采用最近邻分类器进行分类识别。在CK+库和实验室自建库HFUT-FE(HeFei University of Technology-Face Emotion)上的实验结果表明，所提算法具有较高的准确性。
Abstract：To overcome the deficiency of static expression feature, which lacks time information and can not reflect the subtle changes of expression adequately, a dynamic expression recognition method is proposed for non-specific face: the dynamic expression recognition based on Dynamic Time Warping (DTW) and Active Appearance Model (AAM). Firstly, the method of DTW based on local gradient Dual Tree-Complex Wavelet Transform (DT-CWT) dominant direction pattern is used to warp expression sequence. Secondly, using AAM to locate 66 feature points of face image and track them. The changing feature of expression can be obtained by calculating the displacement of corresponding feature points in two adjacent expression sequences image. And using the feature points of neutral face to build the facial geometry model. The matching of facial geometry model can overcome the expression differences between various people. Finally, the nearest neighbor classifier is used for classification and recognition. The experimental results on CK+ database and HeFei University of Technology-Face Emotion (HFUT-FE) database show that the proposed algorithm has a high degree of accuracy.
许良凤, 王家勇,崔婧楠, 胡敏, 张柯柯, 滕文娣. 基于动态时间规整和主动外观模型的动态表情识别[J]. 电子与信息学报, 2018, 40(2): 338-345.
XU Liangfeng, WANG Jiayong, CUI Jingnan, HU Min, ZHANG Keke, TENG Wendi. Dynamic Expression Recognition Based on Dynamic Time Warping and Active Appearance Model. JEIT, 2018, 40(2): 338-345.
YU Jun, WANG Zengfu, and LI Rui. A simultaneous facial motion tracking and expression recognition algorithm[J]. Acta Electronica Sinica, 2015, 43(2): 371-376. doi: 10.3969/ j.issn.0372-2112.2015.02.025.
[3]
JIA J, XU Y, ZHANG S, et al. The facial expression recognition method of random forest based on improved PCA extracting feature[C]. IEEE International Conference on Signal Processing, Communications and Computing, Hong Kong, China. 2016: 1-5. doi: 10.1109/ICSPCC.2016.7753643.
[4]
REN Fuji and HUANG Zhong. Facial expression recognition based on AAM-SIFT and adaptive regional weighting[J]. IEEE Transactions on Electrical & Electronic Engineering, 2015, 10(6): 713-722. doi: 10.1002/tee.22151.
[5]
LI Y, REN F, and HU M. Facial expression recognition based on DT-CWT and weighted inverse cloud model[C]. Modelling, Simulation and Identification 841: Intelligent Systems and Control, Campinas, Brazil. 2016: 380-388. doi: 10.2316/P. 2016.841-010.
[6]
ZHAO Guoying and PIETIKAINEN Matti. Dynamic texture recognition using local binary patterns with an application to facial expressions[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2007, 29(6): 915-928.
[7]
JIANG Bihan, VALSTAR Member, MARTINEZ Brais, et al. A dynamic appearance descriptor approach to facial actions temporal modeling[J]. IEEE Transactions on Cybernetics, 2014, 44(2): 161-174. doi: 10.1109/tcyb.2013.2249063.
[8]
TAHERI Sima, QIU Qiang, and CHELLAPPA Rama. Structure-preserving sparse decomposition for facial expression analysis[J]. IEEE Transactions on Image Processing, 2014, 23(8): 3590-3603. doi: 10.1109/tip.2014. 2331141.
[9]
FAN Xijian and TJAHJADI Tardi. A spatial-temporal framework based on histogram of gradients and optical flow for facial expression recognition in video sequences[J]. Pattern Recognition, 2015, 48(11): 3407-3416. doi: 10.1016/j. patcog.2015.04.025.
[10]
施毅. 基于主动外观模型的人脸表情识别研究[D]. [硕士论文], 上海交通大学, 2012.
SHI Yi. Facial expression recognition based on active appearance models[D]. [Master dissertation], Shanghai Jiao Tong University, 2012.
[11]
CHEON Yeongjae and KIM Daijin. Natural facial expression recognition using differential-AAM and manifold learning[J]. Pattern Recognition, 2009, 42(7): 1340-1350. doi: 10.1016/ j.patcog.2008.10.010.
[12]
LEE Yonghwan, HAN Wuri, KIM Youngseop, et al. Robust emotion recognition algorithm for ambiguous facial expression using optimized AAM and k-NN[J]. International Journal of Security & Its Applications, 2014, 8(5): 203-212. doi: 10.14257/ijsia.2014.8.5.19.
[13]
HIYADI H, ABABSA F, MONTAGNE C, et al. Combination of HMM and DTW for 3D Dynamic gesture recognition using depth only[C]. Informatics in Control, Automation and Robotics 12th International Conference, ICINCO 2015, Colmar, France, 2016: 229-245. doi: 10.1007/978-3-319- 31898-1_13.
FU Ying and GUO Jingyun. Dynamic time warping-based human action recognition[J]. Electronic Measurement Technology, 2014, 37(3): 69-72. doi: 10.3969/j.issn.1002-7300. 2014.03.018.
SONG Yihuan, RAO Xiuqin, and YING Yibing. Apple stem/ calyx and defect discrimination using DT-CWT and LS-SVM [J]. Transactions of the Chinese Society of Agricultural Engineering, 2012, 28(9): 114-118. doi: 10.3969/j.issn.1002- 6819.2012.09.019.
HU Min, ZHU Hong, WANG Xiaohua, et al. Expression recognition method based on gradient Gabor histogram features[J]. Journal of Computer-Aided Design & Computer Graphics, 2013, 25(12): 1856-1861.
[17]
LIU Jun, JING Xiaojun, SUN Songlin, et al. Local gabor dominant direction pattern for face recognition[J]. Chinese Journal of Electronics, 2015, 24(2): 245-250. doi: 10.1049/cje. 2015.04.004.
[18]
HUANG M W, WANG Z W, and YING Z L. A new method for facial expression recognition based on sparse representation plus LBP[C]. IEEE International Congress on Image and Signal Processing, Yantai, China, 2010: 1750-1754. doi: 10.1109/cisp.2010.5647898.
[19]
WANG Zhen and YING Zilu. Facial expression recognition based on local phase quantization and sparse representation [J]. IEEE International Conference on Natural Computation, 2012, 2: 222-225. doi: 10.1109/icnc.2012.6234551.
[20]
ZHANG Shiqing, ZHAO Xiaoming, and LEI Bicheng. Robust facial expression recognition via compressive sensing[J]. Sensors, 2012, 12(3): 3747-3761. doi: 10.3390/s120303747.
