access icon free Improved RGB-D-T based face recognition

Reliable facial recognition systems are of crucial importance in various applications from entertainment to security. Thanks to the deep-learning concepts introduced in the field, a significant improvement in the performance of the unimodal facial recognition systems has been observed in the recent years. At the same time a multimodal facial recognition is a promising approach. This study combines the latest successes in both directions by applying deep learning convolutional neural networks (CNN) to the multimodal RGB, depth, and thermal (RGB-D-T) based facial recognition problem outperforming previously published results. Furthermore, a late fusion of the CNN-based recognition block with various hand-crafted features (local binary patterns, histograms of oriented gradients, Haar-like rectangular features, histograms of Gabor ordinal measures) is introduced, demonstrating even better recognition performance on a benchmark RGB-D-T database. The obtained results in this study show that the classical engineered features and CNN-based features can complement each other for recognition purposes.

Inspec keywords: face recognition; neural nets; visual databases; learning (artificial intelligence); image colour analysis

Other keywords: histogram of Gabor ordinal measures; Haar-like rectangular features; multimodal RGB-depth-thermal based facial recognition; local binary patterns; improved RGB-D-T based face recognition; CNN-based recognition block; histogram of oriented gradients; RGB-D-T database; deep learning convolutional neural networks; multimodal facial recognition; handcrafted features; unimodal facial recognition systems

Subjects: Image recognition; Computer vision and image processing techniques; Neural computing techniques

References

    1. 1)
      • 19. Bowyer, K.W., Chang, K., Flynn, P.: ‘A survey of approaches and challenges in 3d and multi-modal 3D + 2D face recognition’. CVIU, 2006, vol. 101, no. 1, pp. 115.
    2. 2)
      • 10. Goswami, G., Vatsa, M., Singh, R.: ‘RGB-D face recognition with texture and attribute features’, IEEE TIFS, 2014, 9, (10), pp. 16291640.
    3. 3)
      • 3. Zhao, H., Yuen, P.C.: ‘Incremental linear discriminant analysis for face recognition’, IEEE Trans. Syst. Man Cybern. B, 2008, 38, (1), pp. 210221.
    4. 4)
      • 16. Jordi, M., Albiol, A., Paredes, R.: ‘Local deep neural networks for gender recognition’, Pattern Recogn. Lett., 2016, 70, pp. 8086.
    5. 5)
      • 31. Nasrollahi, K., Moeslund, T.B.: ‘Are Haar-like rectangular features for biometric recognition reducible’, In Ruiz-Shulcloper, J., di Baja, G.S. (Eds), ‘Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications’ (Springer, 2013), pp. 334341.
    6. 6)
      • 20. Henry, P., Krainin, M., Herbst, E., et al: ‘Rgb-d mapping: Using kinect style depth cameras for dense 3D modeling of indoor environments’, IJRR, 2012, 31, (5), pp. 647663.
    7. 7)
      • 4. Ahonen, T., Hadid, A., Pietikainen, M.: ‘Face description with local binary patterns: Application to face recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (12), pp. 20372041, doi: 10.1109/TPAMI.2006.244.
    8. 8)
      • 9. Ross, A., Jain, A.K.: ‘Multimodal biometrics: an overview’. Proc. of 12th European Signal Processing Conf., 2004, pp. 12211224.
    9. 9)
      • 33. Yichong, X., Xiao, T., Zhang, J., et al: ‘Scale-invariant convolutional neural networks’. arXiv preprint arXiv:1411.6369, 2014.
    10. 10)
      • 15. Timm, L., Wehner, S., Arras, K.O.: ‘Real-time full-body human gender recognition in (RGB)-D data’. 2015 IEEE Int. Conf. on Robotics and Automation (ICRA), IEEE, 2015.
    11. 11)
      • 1. Sun, N., Wang, H., Ji, Z.-h., et al: ‘An efficient algorithm for kernel two-dimensional principal component analysis. Neural computing and applications’, 2008, 17, (1), pp. 5964, doi: 10.1007/s00521-007-0111-0.
    12. 12)
      • 27. Taigman, Y., Yang, M., Ranzato, M., et al: ‘Web-scale training for face identification’. CoRR, abs/1406.5266, 2014.
    13. 13)
      • 23. Kakadiaris, I.A., Passalis, G., Theoharis, T., et al: ‘Multimodal face recognition: Combination of geometry with physiological information’. Proc. CVPR, 2005.
    14. 14)
      • 21. Holz, D., Holzer, S., Rusu, R., et al: ‘Real-time plane segmentation using RGB-D cameras’. Proc. Robot Soccer world Cup XV, 2012.
    15. 15)
      • 30. Phillips, P.J., Flynn, P.J., Scruggs, T., et al: ‘Overview of the face recognition grand challenge’, In (Eds), ‘IEEE computer society conference on computer vision and pattern recognition, IEEE’ (2005), vol. 1, pp. 947954.
    16. 16)
      • 8. Taigman, Y., Yang, M., Ranzato, M., et al, Deepface: ‘Closing the gap to human-level performance in face verification’. 2014 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2014, pp. 17011708, doi: 10.1109/CVPR.2014.220.
    17. 17)
      • 12. Segundo, M.P., Sarkar, S., Goldgof, D., et al: ‘Continuous 3d face authentication using RGB-D cameras’. Proc. CVRPW, 2013.
    18. 18)
      • 24. Lawrence, S., Giles, C.L., Tsoi, A.C.: ‘Convolutional neural networks for face recognition’. 1996 Conf. on Computer Vision and Pattern Recognition (CVPR ’96), IEEE Computer Society, 1996.San Francisco, CA, USA, June 18–20, 1996, pp. 217222, ISBN 0-8186-7258-7.
    19. 19)
      • 2. Zhang, D., Zhou, Z.-H., Chen, S.: ‘Diagonal principal component analysis for face recognition’, Pattern Recogn., 2006, 39, (1), pp. 140142, ISSN 0031-3203.
    20. 20)
      • 34. Jiquan, N., Chen, Z., Chia, D., et al: ‘Tiled convolutional neural networks’. NIPS, 2010, pp. 12791287.
    21. 21)
      • 14. Hsu, G.S.J., Liu, Y.L., Peng, H.C., et al: ‘RGB-D-based face reconstruction and recognition’, IEEE Trans. Inf. Forensics Sec., 2014, 9, (12), pp. 21102118.
    22. 22)
      • 7. Chai, Z., Sun, Z., Mendez-Vazquez, H., et al: ‘Gabor ordinal measures for face recognition’, IEEE Trans. Inf. Forensics and Sec., 2014, 9, (1), pp. 1426.
    23. 23)
      • 13. Zheng, Y., Elmaghraby, A.: ‘A brief survey on multispectral face recognition and multimodal score fusion’. ISSPIT, 2011, pp. 543550.
    24. 24)
      • 28. Phillips, P.J., Moon, H., Rizvi, S.A., et al: ‘The ferret evaluation methodology for face-recognition algorithms’, IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22, (10), pp. 10901104.
    25. 25)
      • 29. Martinez, A.M.: ‘The AR face database’. CVC Technical Report, 24, 1998.
    26. 26)
      • 32. Nair, V., Hinton, G.E.: ‘Rectified Linear Units Improve Restricted Boltzmann Machines’. Proc. of Int. Conf. on Machine Learning, 2010.
    27. 27)
      • 22. Ramey, A., González-Pacheco, V., Salichs, M.A.: ‘Integration of a low-cost rgb-d sensor in a social robot for gesture recognition’. Proc. ACM/IEEE HRI, 2011.
    28. 28)
      • 6. Nasrollahi, K., Moeslund, T.B.: ‘Haar-like features for robust real-time face recognition’. 2013 20th IEEE Int. Conf. on Image Processing (ICIP), September 2013, pp. 30733077, doi: 10.1109/ICIP.2013.6738633.
    29. 29)
      • 11. Li, B.Y.L., Mian, A.S., Liu, W., et al: ‘Using kinect for face recognition under varying poses, expressions, illumination and disguise’. In Proc. WACV, 2013.
    30. 30)
      • 26. Goswami, G., Bhardwaj, R., Singh, R., et al: ‘Memorability augmented deep learning for video face recognition’. , 2014 IEEE International Joint Conf. on Biometrics (IJCB), September 2014, pp. 17, doi: 10.1109/BTAS.2014.6996299.
    31. 31)
      • 17. Nikisins, O., Nasrollahi, K., Greitans, M., et al: ‘RGB-D-T based face recognition’. 22nd Int. Conf. on Pattern Recognition, ICPR 2014, IEEE, 2014, Stockholm, Sweden, August 24–28, 2014, pp. 17161721.
    32. 32)
      • 25. Huang, G.B., Ramesh, M., Berg, T., et al: ‘Labeled faces in the wild: A database for studying face recognition in unconstrained environments’. Technical Report 07–49, University of Massachusetts, Amherst, 2007.
    33. 33)
      • 5. Dalal, N., Triggs, B.: ‘Histograms of oriented gradients for human detection’. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005. CVPR 2005.June 2005, vol. 1, pp. 886893.
    34. 34)
      • 18. Bebis, G., Gyaourova, A., Singh, S., et al: ‘Face recognition by fusing thermal infrared and visible imagery’, IVC, 2006, 24, (7), pp. 727742.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-bmt.2015.0057
Loading

Related content

content/journals/10.1049/iet-bmt.2015.0057
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading