© The Institution of Engineering and Technology
A novel framework, named intra-class variation reduced features-based manifold regularisation dictionary pair learning model, is presented for solving facial expression recognition (FER) tasks. Since a query face and its corresponding image with intra-class variations (e.g. identity and illumination) are similar in appearance, the authors generate intra-class variation reduced features (IVRF) from the difference between a query face image and its corresponding estimated image of each expression class. IVRF can reduce negative influence from the intra-class variations and make their model robust to intra-class variations. Furthermore, a manifold regularisation term is incorporated into the dictionary pair learning model, which leads to a smoothly varying sparse representation. Their model fully takes advantage of the geometrical structure of data, which benefits the FER task. The experimental results on two public databases verify the effectiveness and superiority of their method and indicate its promising capability in expression discrimination.
References
-
-
1)
-
24. Yang, M., Zhang, L., Feng, X., et al: ‘Fisher discrimination dictionary learning for sparse representation’. Proc. IEEE Int. Conf. Computer Vision Proc, Barcelona, Spain, November 2011, pp. 543–550.
-
2)
-
40. Belkin, M., Niyogi, P., Sindhwani, V.: ‘Manifold regularization: a geometric framework for learning from labeled and unlabeled examples’, J. Mach. Learn. Res., 2006, 7, pp. 2399–2434.
-
3)
-
19. Shan, C., Gong, S., McOwan, P.W.: ‘Facial expression recognition based on local binary patterns: a comprehensive study’, J. Image Vis. Comput., 2009, 27, (6), pp. 803–816.
-
4)
-
16. Tong, Y., Chen, J., Ji, Q.: ‘A unified probabilistic framework for spontaneous facial action modeling and understanding’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32, (2), pp. 258–273.
-
5)
-
7. Happy, S.L., Routray, A.: ‘Automatic facial expression recognition using features of salient facial patches’, IEEE Trans. Affective Comput., 2015, 6, (1), pp. 1–12.
-
6)
-
42. Boyd, S., Parikh, N., Chu, E., et al: ‘Distributed optimization and statistical learning via the alternating direction method of multipliers’, J. Found. Trends Mach. Learn., 2011, 3, (1), pp. 1–122.
-
7)
-
31. Chen, D., Cao, X., Wen, F., et al: ‘Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, Portland, USA, June 2013, pp. 3025–3032.
-
8)
-
14. Bartlett, M.S., Hager, J.C., Ekman, P., et al: ‘Measuring facial expressions by computer image analysis’, Psychophysiology, 1999, 36, (2), pp. 253–263.
-
9)
-
20. Zafeiriou, S., Petrou, M.: ‘Sparse representation for facial expression recognition via l1 optimization’. Proc. IEEE Int. Conf. Computer Vision Pattern Recognition, San Francisco, USA, June 2010, pp. 32–39.
-
10)
-
4. Valstar, M.F., Mehu, M., Jiang, B., et al: ‘Meta-Analysis of the first facial expression recognition challenge’, IEEE Trans. Syst. Man Cybern. Part B, 2012, 42, (4), pp. 966–979.
-
11)
-
33. Chen, J., Shan, S., He, C., et al: ‘WLD: a robust local image descriptor’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32, (9), p. 1705.
-
12)
-
15. Kanade, T., Cohn, J.F., Tian, Y.: ‘Recognizing action units for facial expression analysis’, IEEE Trans. Pattern Anal. Mach. Intell., 2001, 23, (2), pp. 97–115.
-
13)
-
9. Rinn, W.E.: ‘The neuropsychology of facial expression: a review of the neurological and psychological mechanisms for producing facial expressions’, Psychol. Bull., 1984, 95, (1), pp. 52–77.
-
14)
-
37. Wright, J., Yang, A.Y., Ganesh, A., et al: ‘Robust face recognition via sparse representation’, IEEE Trans. Pattern Anal. and Mach. Intell., 2009, 31, (2), pp. 210–227.
-
15)
-
22. Yang, P., Liu, Q., Metaxas, D.N.: ‘Exploring facial expressions with compositional features’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, San Francisco, USA, June 2010, pp. 2638–2644.
-
16)
-
36. Wang, Z., Yang, J., Nasrabadi, N., et al: ‘A max-margin perspective on sparse representation-based classification’. Proc. IEEE Int. Conf. Computer Vision, Sydney, Australia, June 2013, pp. 1217–1224.
-
17)
-
34. Ojala, T., Pietikainen, M., Maenpaa, T.: ‘Multiresolution gray-scale and rotation invariant texture classification with local binary patterns’, IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24, (7), pp. 971–987.
-
18)
-
11. Gu, S., Zhang, L., Zuo, W., et al: ‘Projective dictionary pair learning for pattern classification’, NIPS'14 Proceedings of the 27th International Conference on Neural Information Processing Systems, 2014, 1, pp. 793–801.
-
19)
-
8. Taheri, S., Qiu, Q., Chellappa, R.: ‘Structure-preserving sparse decomposition for facial expression analysis’, IEEE Trans. Image Process., 2014, 23, (8), pp. 3590–3603.
-
20)
-
10. Duchenne, G.B.: ‘Mecanisme de la Physionomie Humaine’ (Renouard, Paris, France, 1862).
-
21)
-
46. Zhang, L., Yang, M., Feng, X.: ‘Sparse representation or collaborative representation: which helps face recognition?’. Proc. IEEE Int. Conf. Computer Vision, Barcelona, Spain, November 2011, pp. 471–478.
-
22)
-
23. Ramirez, , Sprechmann, P., Sapiro, G.: ‘Classification and clustering via dictionary learning with structured incoherence and shared features’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, San Francisco, USA, June 2010, pp. 3501–3508.
-
23)
-
43. Xie, S., Hu, H.: ‘Facial expression recognition with FRR-CNN’, Electron. Lett., 2017, 53, (4), pp. 235–237.
-
24)
-
13. Turk, M., Pentland, A.: ‘Eigenfaces for recognition’, J. Cogn. Neurosci., 1991, 3, (1), pp. 71–86.
-
25)
-
45. Lucey, P., Cohn, J.F., Kanade, T., et al: ‘The extended Cohn–Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression’. Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition-Workshops, 2010, pp. 94–101.
-
26)
-
38. Belkin, M., Niyogi, P.: ‘Laplacian eigenmaps and spectral techniques for embedding and clustering’. Proc. Int. Conf. Neural Information Processing Systems, Vancouver, Canada, December 2001, 6, pp. 585–591.
-
27)
-
39. Cai, D., He, X., Wu, X., et al: ‘Non-negative matrix factorization on manifold’. Proc. IEEE Int. Conf. Data Mining, Pisa, Italy, December 2008, pp. 63–72.
-
28)
-
47. Chen, B.C., Chen, C.S., Hsu, W.: ‘Review and implementation of high-dimensional local binary patterns and its application to face recognition’ (Institute of Information Science, Academia Sinica, , 2014).
-
29)
-
25. Gao, S., Tsang, W.H., Ma, Y.: ‘Learning category-specific dictionary and shared dictionary for fine-grained image categorization’, IEEE Trans. Image Process., 2014, 23, (2), pp. 623–634.
-
30)
-
32. Liao, S., Law, M.W.K., Chung, A.C.S.: ‘Dominant local binary patterns for texture classification’, IEEE Trans. Image Process., 2009, 18, (5), pp. 1107–1118.
-
31)
-
21. Lee, S.H., Plataniotis, K.N.K., Yong, M.R.: ‘Intra-class variation reduction using training expression images for sparse representation based facial expression recognition’, IEEE Trans. Affective Comput., 2014, 5, (3), pp. 340–351.
-
32)
-
26. Zhang, Q., Li, B.: ‘Discriminative K-SVD for dictionary learning in face recognition’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, San Francisco, USA, June 2010, pp. 2691–2698.
-
33)
-
18. Guo, G., Dyer, C.R.: ‘Learning from examples in the small sample case: face expression recognition’, IEEE Trans. Syst. Man Cybern. Part B, 2005, 35, (3), pp. 477–488.
-
34)
-
2. Anderson, K., McOwan, P.W.: ‘A real-time automated system for the recognition of human facial expressions’, IEEE Trans. Syst. Man Cybern. Part B, 2006, 36, (1), pp. 96–105.
-
35)
-
6. Tian, Y., Kanade, T., Cohn, J.F.: ‘Facial expression recognition’ (Springer-Verlag, New York, NY, USA).
-
36)
-
27. Jiang, Z., Lin, Z., Davis, L.S.: ‘Label consistent K-SVD: learning a discriminative dictionary for recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (11), pp. 2651–2664.
-
37)
-
3. Ekman, P., Friesen, W.: ‘Measuring facial movement with the facial action coding system’ (P Ekman Emotion in the Human Face, 1982).
-
38)
-
5. Valstar, M.F., Pantic, M.: ‘Biologically vs. Logic inspired encoding of facial actions and emotions in video’. Proc. IEEE Int. Conf. Multimedia and Expo, Ontario, Toronto, July 2006, pp. 325–328.
-
39)
-
1. Kotsia, I, Pitas, I.: ‘Facial expression recognition in image sequences using geometric deformation features and support vector machines’, IEEE Trans. Image Process., 2007, 16, (1), pp. 172–187.
-
40)
-
17. Sandbach, G., Zafeiriou, S., Pantic, M.: ‘Binary pattern analysis for 3D facial action unit detection’. Proc. Conf. Brit. Mach. Vis., Surrey, UK, 2012, , pp. 1–12.
-
41)
-
12. He, X., Yan, S., Hu, Y., et al: ‘Face recognition using Laplacianfaces’, IEEE Trans. Pattern Anal. Mach. Intell., 2005, 27, (3), pp. 328–340.
-
42)
-
28. Zeng, X., Bian, W., Liu, W., et al: ‘Dictionary pair learning on Grassmann manifolds for image denoising’, IEEE Trans. Image Process., 2015, 24, (11), pp. 4556–4569.
-
43)
-
30. Xiong, X., Torre, F.D.L.: ‘Supervised descent method and its applications to face alignment’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, Portland, USA, June 2013, pp. 532–539.
-
44)
-
29. Viol, P., Jones, M.J..: ‘Robust real-time face detection’, Int. J. Comput. Vis., 2004, 57, (2), pp. 137–154.
-
45)
-
41. Zhou, D., Bousquet, O., Lal, T.N., et al: ‘Learning with local and global consistency’, J. Adv. Neural Inf. Process. Syst., 2004, 16, (16), pp. 321–328.
-
46)
-
44. Kanade, T., Cohn, J.F., Tian, Y.: ‘Comprehensive database for facial expression analysis’. Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, Washington, DC, USA, May 2002, pp. 46–53.
-
47)
-
35. Zhang, L., Yang, M., Feng, X.C., et al: ‘Collaborative representation based classification for face recognition’, .
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2017.0422
Related content
content/journals/10.1049/iet-cvi.2017.0422
pub_keyword,iet_inspecKeyword,pub_concept
6
6