Fusing the facial temporal information in videos for face recognition

Fusing the facial temporal information in videos for face recognition

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Computer Vision — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Face recognition is a challenging and innovative research topic in the present sophisticated world of visual technology. In most of the existing approaches, the face recognition from the still images is affected by intra-personal variations such as pose, illumination and expression which degrade the performance. This study proposes a novel approach for video-based face recognition due to the availability of large intra-personal variations. The feature vector based on the normalised semi-local binary patterns is obtained for the face region. Each frame is matched with the signature of the faces in the database and a rank list is formed. Each ranked list is clustered and its reliability is analysed for re-ranking. To characterise an individual in a video, multiple re-ranked lists across the multiple video frames are fused to form a video signature. This video signature embeds diverse intra-personal and temporal variations, which facilitates in matching two videos with large variations. For matching two videos, their video signatures are compared using Kendall-Tau distance. The developed methods are deployed on the YouTube and ChokePoint videos, and they exhibit significant performance improvement owing to their approach when compared with the existing techniques.


    1. 1)
      • 1. Hassaballah, M., Aly, S.: ‘Face recognition: challenges, achievements and future directions computer vision’, IET Comput. Vis., 2015, 9, (4), pp. 614626.
    2. 2)
      • 2. Faraji, M.R., Xiaojun, Q.: ‘Face recognition under varying illumination based on adaptive homomorphic eight local directional patterns’, IET Comput. Vis., 2015, 9, (3), pp. 390399.
    3. 3)
      • 3. Ochoa, V., Nolazco-Flores, J.A., Barron-Cano, O., et al: ‘Addressing the illumination challenge in two-dimensional face recognition: a survey’, IET Comput. Vis.,2015, 9, (6), pp. 978992.
    4. 4)
      • 4. Gary, B.H., Honglak, L., Ann, A.: ‘Learning hierarchical representations for face verification with convolutional deep belief networks’. IEEE Conf. on Computer Vision and Pattern Recognition, 2012, pp. 25182525.
    5. 5)
      • 5. Zhenyao, Z., Ping, L., Xiaogang, W., et al: ‘Deep learning multi-view representation for face recognition’. Computer Vision and Pattern Recognition, 2014.
    6. 6)
      • 6. Barr, J.R., Bowyer, K.W., Flynn, P.J.: ‘Face recognition from video: a review’, J. Pattern Recognit. Artif. Intell., 2012, 26, (5), pp. 156.
    7. 7)
      • 7. Hu, Y., Mian, A.S., Owens, R.: ‘Sparse approximated nearest points for image set classification’. IEEE Conf. on Computer Vision and Pattern Recognition, 2011, pp. 2740.
    8. 8)
      • 8. Yang, M., Zhu, P., Van, G.L., et al: ‘Face recognition based on regularized nearest points between image sets’. Proc. IEEE Conf. on Automatic Face and Gesture Recognition, 2013, pp. 17.
    9. 9)
      • 9. Wolf, L., Hassner, T., Maoz, I.: ‘Face recognition in unconstrained videos with matched background similarity’. Int. Conf. on Computer Vision and Pattern Recognition, 2011, pp. 529534.
    10. 10)
      • 10. Cui, Z., Shan, S., Zhang, H., et al: ‘Image sets alignment for video-based face recognition’. Proc. Int. Conf. on Computer Vision and Pattern Recognition, 2012, pp. 26262633.
    11. 11)
      • 11. Pengfei, Z., Wangmeng, Z., Lei, Z., et al: ‘Image set-based collaborative representation for face recognition’, IEEE Trans. Inf. Forensics Sec., 2014, 9, (7), pp. 11201132.
    12. 12)
      • 12. Wang, R., Shan, S., Chen, X., et al: ‘Manifold-manifold distance and its application to face recognition with image sets’, IEEE Trans. Image Process., 2012, 21, (10), pp. 44664479.
    13. 13)
      • 13. Zhou, S., Krueger, V., Chellappa, R.: ‘Probabilistic recognition of human faces from video’, Comput. Vis. Image Underst., 2003, 91, (1/2), pp. 214245.
    14. 14)
      • 14. Kim, M., Kumar, S., Pavlovic, V., et al: ‘Face tracking and recognition with visual constraints in real-world videos’. Proc. Int. Conf. on Computer Vision and Pattern Recognition, 2008, pp. 18.
    15. 15)
      • 15. Liu, X., Chen, T.: ‘Video-based face recognition using adaptive hidden Markov models’. Proc. Int. Conf. on Computer Vision and Pattern Recognition, 2003, pp. 340345.
    16. 16)
      • 16. Jain, A., Nandakumar, k., Ross, A.: ‘Score normalization in multimodal biometric systems’, Pattern Recognit., 2005, 38, (12), pp. 22702285.
    17. 17)
      • 17. Bhatt, H.S., Singh, R., Vatsa, M.: ‘On rank aggregation for face recognition from videos’. Proc. Int. Conf. on Image Processing, 2013, pp. 15.
    18. 18)
      • 18. Gaopeng, G., Huang, D., Wang, Y.: ‘Video face recognition via combination of real-time local features and temporal-spatial cues’, IET Comput. Vis., 2014, 8, (4), pp. 347357.
    19. 19)
      • 19. Cui, Z., Li, W., Xu, D., et al: ‘Fusing robust face region descriptors via multiple metric learning for face recognition in the wild’. IEEE CVPR, 2013, pp. 35543561.
    20. 20)
      • 20. O'Toole, A.J., Roark, D.A., Abdi, H.: ‘Recognizing moving faces: a psychological and neural synthesis’. Trends in Cognitive Sciences, 2002, pp. 261266.
    21. 21)
      • 21. Viola, P., Michael, J.: ‘Robust real-time object detection’, Int. J. Comput. Vis., 2004, 57, (2), pp. 137154.
    22. 22)
      • 22. Belhumeur, P., Hespanha, J., Kriegman, D.: ‘Eigenfaces vs fisherfaces: recognition using class specific linear projection’, IEEE Trans. Pattern Anal. Mach. Intell., 1997, 20, (7), pp. 711720.
    23. 23)
      • 23. Lee, T.S.: ‘Image representation using 2d Gabor wavelets’, IEEE Trans. PAMI, 1996, 18, (10), pp. 959971.
    24. 24)
      • 24. Huang, D., Shan, C., Ardabilian, M., et al: ‘Local binary patterns and its application to facial image analysis: a survey’, IEEE Trans. Syst. Man Cybern., 2011, 41, (6), pp. 765781.
    25. 25)
      • 25. Suruliandi, A., Meena, K., Reena, R.: ‘Local binary pattern and its derivatives for face recognition’, IET Comput. Vis.,2012, 6, (5), pp. 480488.
    26. 26)
      • 26. Kyungjoong, J., Choi, J., Jang, G.: ‘Semi-local structure patterns for robust face detection’, IEEE Signal Process. Lett., 2015, 22, (9), pp. 14001403.
    27. 27)
      • 27. Kim, M., Kumar, S., Pavlovic, V., et al: ‘Face tracking and recognition with visual constraints in real-world videos’. IEEE Conf. on Computer Vision and Pattern Recognition, 2008.
    28. 28)
      • 28. Wong, Y., Chen, S., Mau, S., et al: ‘Patch-based probabilistic image quality assessment for face selection and improved video-based face recognition’. IEEE Biometrics Workshop, Computer Vision and Pattern Recognition Workshops, 2011, pp. 8188.
    29. 29)
      • 29. Zhang, J., Gao, J., Zhou, M., et al: ‘Improving the effectiveness of information retrieval with clustering and fusion’, J. Comput. Linguist. Chin. Lang. Process., 2001, 6, (1), pp. 109125.
    30. 30)
      • 30. Sculley, D.: ‘Rank aggregation for similar items’. Proc. Int. Conf. on Data Mining, 2007, pp. 587592.
    31. 31)
      • 31. Sun, Y., Chen, Y., Wang, X., et al: ‘Deep learning face representation by joint identification-verification’. Proc. NIPS, 2014, pp. 19881996.
    32. 32)
      • 32. Sun, Y., Wang, X., Tang, X.: ‘Deep learning face representation from predicting 10,000 classes’. Proc. IEEE CVPR, 2014, pp. 18911898.

Related content

This is a required field
Please enter a valid email address