© The Institution of Engineering and Technology
Reliable facial recognition systems are of crucial importance in various applications from entertainment to security. Thanks to the deep-learning concepts introduced in the field, a significant improvement in the performance of the unimodal facial recognition systems has been observed in the recent years. At the same time a multimodal facial recognition is a promising approach. This study combines the latest successes in both directions by applying deep learning convolutional neural networks (CNN) to the multimodal RGB, depth, and thermal (RGB-D-T) based facial recognition problem outperforming previously published results. Furthermore, a late fusion of the CNN-based recognition block with various hand-crafted features (local binary patterns, histograms of oriented gradients, Haar-like rectangular features, histograms of Gabor ordinal measures) is introduced, demonstrating even better recognition performance on a benchmark RGB-D-T database. The obtained results in this study show that the classical engineered features and CNN-based features can complement each other for recognition purposes.
References
-
-
1)
-
19. Bowyer, K.W., Chang, K., Flynn, P.: ‘A survey of approaches and challenges in 3d and multi-modal 3D + 2D face recognition’. CVIU, 2006, vol. 101, no. 1, pp. 1–15.
-
2)
-
10. Goswami, G., Vatsa, M., Singh, R.: ‘RGB-D face recognition with texture and attribute features’, IEEE TIFS, 2014, 9, (10), pp. 1629–1640.
-
3)
-
3. Zhao, H., Yuen, P.C.: ‘Incremental linear discriminant analysis for face recognition’, IEEE Trans. Syst. Man Cybern. B, 2008, 38, (1), pp. 210–221.
-
4)
-
16. Jordi, M., Albiol, A., Paredes, R.: ‘Local deep neural networks for gender recognition’, Pattern Recogn. Lett., 2016, 70, pp. 80–86.
-
5)
-
31. Nasrollahi, K., Moeslund, T.B.: ‘Are Haar-like rectangular features for biometric recognition reducible’, In Ruiz-Shulcloper, J., di Baja, G.S. (Eds), ‘Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications’ (Springer, 2013), pp. 334–341.
-
6)
-
20. Henry, P., Krainin, M., Herbst, E., et al: ‘Rgb-d mapping: Using kinect style depth cameras for dense 3D modeling of indoor environments’, IJRR, 2012, 31, (5), pp. 647–663.
-
7)
-
4. Ahonen, T., Hadid, A., Pietikainen, M.: ‘Face description with local binary patterns: Application to face recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (12), pp. 2037–2041, .
-
8)
-
9. Ross, A., Jain, A.K.: ‘Multimodal biometrics: an overview’. Proc. of 12th European Signal Processing Conf., 2004, pp. 1221–1224.
-
9)
-
33. Yichong, X., Xiao, T., Zhang, J., et al: ‘Scale-invariant convolutional neural networks’. arXiv preprint arXiv:1411.6369, 2014.
-
10)
-
15. Timm, L., Wehner, S., Arras, K.O.: ‘Real-time full-body human gender recognition in (RGB)-D data’. 2015 IEEE Int. Conf. on Robotics and Automation (ICRA), IEEE, 2015.
-
11)
-
1. Sun, N., Wang, H., Ji, Z.-h., et al: ‘An efficient algorithm for kernel two-dimensional principal component analysis. Neural computing and applications’, 2008, 17, (1), pp. 59–64, .
-
12)
-
27. Taigman, Y., Yang, M., Ranzato, M., et al: ‘Web-scale training for face identification’. CoRR, , 2014.
-
13)
-
23. Kakadiaris, I.A., Passalis, G., Theoharis, T., et al: ‘Multimodal face recognition: Combination of geometry with physiological information’. Proc. CVPR, 2005.
-
14)
-
21. Holz, D., Holzer, S., Rusu, R., et al: ‘Real-time plane segmentation using RGB-D cameras’. Proc. Robot Soccer world Cup XV, 2012.
-
15)
-
30. Phillips, P.J., Flynn, P.J., Scruggs, T., et al: ‘Overview of the face recognition grand challenge’, In (Eds), ‘IEEE computer society conference on computer vision and pattern recognition, IEEE’ (2005), vol. 1, pp. 947–954.
-
16)
-
8. Taigman, Y., Yang, M., Ranzato, M., et al, : ‘Closing the gap to human-level performance in face verification’. 2014 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), June 2014, pp. 1701–1708, .
-
17)
-
12. Segundo, M.P., Sarkar, S., Goldgof, D., et al: ‘Continuous 3d face authentication using RGB-D cameras’. Proc. CVRPW, 2013.
-
18)
-
24. Lawrence, S., Giles, C.L., Tsoi, A.C.: ‘Convolutional neural networks for face recognition’. 1996 Conf. on Computer Vision and Pattern Recognition (CVPR ’96), IEEE Computer Society, 1996.San Francisco, CA, USA, June 18–20, 1996, pp. 217–222, .
-
19)
-
2. Zhang, D., Zhou, Z.-H., Chen, S.: ‘Diagonal principal component analysis for face recognition’, Pattern Recogn., 2006, 39, (1), pp. 140–142, .
-
20)
-
34. Jiquan, N., Chen, Z., Chia, D., et al: ‘Tiled convolutional neural networks’. NIPS, 2010, pp. 1279–1287.
-
21)
-
14. Hsu, G.S.J., Liu, Y.L., Peng, H.C., et al: ‘RGB-D-based face reconstruction and recognition’, IEEE Trans. Inf. Forensics Sec., 2014, 9, (12), pp. 2110–2118.
-
22)
-
7. Chai, Z., Sun, Z., Mendez-Vazquez, H., et al: ‘Gabor ordinal measures for face recognition’, IEEE Trans. Inf. Forensics and Sec., 2014, 9, (1), pp. 14–26.
-
23)
-
13. Zheng, Y., Elmaghraby, A.: ‘A brief survey on multispectral face recognition and multimodal score fusion’. ISSPIT, 2011, pp. 543–550.
-
24)
-
28. Phillips, P.J., Moon, H., Rizvi, S.A., et al: ‘The ferret evaluation methodology for face-recognition algorithms’, IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22, (10), pp. 1090–1104.
-
25)
-
29. Martinez, A.M.: ‘The AR face database’. , 1998.
-
26)
-
32. Nair, V., Hinton, G.E.: ‘Rectified Linear Units Improve Restricted Boltzmann Machines’. Proc. of Int. Conf. on Machine Learning, 2010.
-
27)
-
22. Ramey, A., González-Pacheco, V., Salichs, M.A.: ‘Integration of a low-cost rgb-d sensor in a social robot for gesture recognition’. Proc. ACM/IEEE HRI, 2011.
-
28)
-
6. Nasrollahi, K., Moeslund, T.B.: ‘Haar-like features for robust real-time face recognition’. 2013 20th IEEE Int. Conf. on Image Processing (ICIP), September 2013, pp. 3073–3077, .
-
29)
-
11. Li, B.Y.L., Mian, A.S., Liu, W., et al: ‘Using kinect for face recognition under varying poses, expressions, illumination and disguise’. In Proc. WACV, 2013.
-
30)
-
26. Goswami, G., Bhardwaj, R., Singh, R., et al: ‘Memorability augmented deep learning for video face recognition’. , 2014 IEEE International Joint Conf. on Biometrics (IJCB), September 2014, pp. 1–7, .
-
31)
-
17. Nikisins, O., Nasrollahi, K., Greitans, M., et al: ‘RGB-D-T based face recognition’. 22nd Int. Conf. on Pattern Recognition, ICPR 2014, IEEE, 2014, Stockholm, Sweden, August 24–28, 2014, pp. 1716–1721.
-
32)
-
25. Huang, G.B., Ramesh, M., Berg, T., et al: ‘Labeled faces in the wild: A database for studying face recognition in unconstrained environments’. Technical Report 07–49, University of Massachusetts, Amherst, 2007.
-
33)
-
5. Dalal, N., Triggs, B.: ‘Histograms of oriented gradients for human detection’. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005. CVPR 2005.June 2005, vol. 1, pp. 886–893.
-
34)
-
18. Bebis, G., Gyaourova, A., Singh, S., et al: ‘Face recognition by fusing thermal infrared and visible imagery’, IVC, 2006, 24, (7), pp. 727–742.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-bmt.2015.0057
Related content
content/journals/10.1049/iet-bmt.2015.0057
pub_keyword,iet_inspecKeyword,pub_concept
6
6