Facial expression recognition based on geometric and optical flow features in colour image sequences

Facial expression recognition based on geometric and optical flow features in colour image sequences

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Computer Vision — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Facial expression recognition is a useful feature in modern human computer interaction (HCI). In order to build efficient and reliable recognition systems, face detection, feature extraction and classification have to be robustly realised. Addressing the latter two issues, this work proposes a new method based on geometric and transient optical flow features and illustrates their comparison and integration for facial expression recognition. In the authors’ method, photogrammetric techniques are used to extract three-dimensional (3-D) features from every image frame, which is regarded as a geometric feature vector. Additionally, optical flow-based motion detection is carried out between consecutive images, what leads to the transient features. Artificial neural network and support vector machine classification results demonstrate the high performance of the proposed method. In particular, through the use of 3-D normalisation and colour information, the proposed method achieves an advanced feature representation for the accurate and robust classification of facial expressions.


    1. 1)
      • M. Pantic , S.Z. Li . (2009) Facial expression recognition, Encyclopedia of biometrics.
    2. 2)
    3. 3)
    4. 4)
      • Wang, J., Yin, L., Wei, X., Sun, Y.: `3D facial expression recognition based on primitive surface feature distribution', IEEE Int. Conf. on Computer Vision and Pattern Recognition, 2006, p. 1399–1406.
    5. 5)
      • Valstar, M.F., Pantic, M.: `Fully automatic facial action unit detection and temporal analysis', Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, 2006.
    6. 6)
    7. 7)
      • Bartlett, S., Littlewort, G., Frank, G., Lainscsek, C., Fasel, I., Movellan, J.: `Fully automatic facial action recognition in spontaneous behavior', Proc. Conf. Automatic Face & Gesture Recognition, 2006, p. 223–230.
    8. 8)
      • I. Pandzic , R. Forchheimer . (2002) MPEG-4 facial animation: the standard, implementation and applications.
    9. 9)
      • Santana, M.C., Lorenzo-Navarro, J., Déniz-Suárez, O., Isern González, J., Falcón-Martel, A.: `Multiple face detection at different resolutions for perceptual user interfaces', IbPRIA, 2005, p. 445–452.
    10. 10)
      • A. Al-Hamadi , R. Niese , B. Michaelis . (2003) A robust approach for contour extraction and tracking of moving objects in video sequences.
    11. 11)
      • C. McGlone , E. Mikhail , J. Bethel . (2004) Manual of photogrammetry.
    12. 12)
      • J. Albertz , W. Kreiling . (1989) Photogrammetric guide.
    13. 13)
    14. 14)
      • Blanz, V., Vetter, T.: `A morphable model for the synthesis of 3D faces', SIGGRAPH Proc., 1999, p. 187–194.
    15. 15)
      • Albrecht, P., Michaelis, B.: `Stereo photogrammetry with improved spatial resolution', ICPR, 1998, p. 845.
    16. 16)
      • Niese, R., Al-Hamadi, A., Michaelis, B.: `A stereo and color-based method for face pose estimation and facial feature extraction', ICPR, 2006, pp. 299–302.
    17. 17)
      • Rusinkiewicz, S., Levoy, M.: `Efficient variants of the ICP algorithm', Proc. Third Int. Conf. on 3D Digital Imaging & Modeling, 2001, p. 145–152.
    18. 18)
      • Schindler, K., van Gool, L.: `Action snippets: how many frames does human action recognition apply?', Proc. IEEE Conf. on Computer Vision and Pattern recognition (CVPR'08), 22–24 June 2008, Anchorage, AK, USA.
    19. 19)
      • Lucas, B., Kanade, T.: `An iterative image registration technique with an application to stereo vision', Seventh Int. Joint Conf. on Artificial Intelligence, 1981, p. 674–679.
    20. 20)
    21. 21)
    22. 22)
    23. 23)
      • R. Niese , A. Al-Hamadi , B. Michaelis . Nearest neighbor classification for emotion recognition in stereo image sequences. ISAST Trans. Electron. Signal Process. , 1 , 88 - 94
    24. 24)
      • S. Haykin . Neural networks: a comprehensive foundation.
    25. 25)
      • N. Cristianini , J.S. Taylor . (2001) An Introduction to Support Vector Machines and other kernel based learning methods.
    26. 26)
      • T.F. Wu , C.J. Lin . Probability estimates for multi-class classification by pairwise coupling. J. Mach. Learn. Res. , 975 - 1005
    27. 27)
      • T. Kohonen . (1995) Self-organizing maps.
    28. 28)
      • H. Ritter , T. Martinez , K. Schulten . (1992) Neural computation and self-organizing maps: an introduction.
    29. 29)
      • Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: `A high-resolution 3D dynamic facial expression database', IEEE Int. Conf. on Automatic Face and Gesture Recognition (FG'08), 2008, p. 1–6.
    30. 30)
      • R. Kohavi , P. Foster . Glossary of terms. Appl. Mach. Learn. Knowl. Discov. Process. (Special Issue) , 23 , 271 - 274

Related content

This is a required field
Please enter a valid email address