Multimodal 2D–3D face recognition using local descriptors: pyramidal shape map and structural context

Multimodal 2D–3D face recognition using local descriptors: pyramidal shape map and structural context

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Biometrics — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

In this study, the authors propose a local descriptor based multimodal approach to improve face recognition performance. Pre-processing is done to smooth, resample, and register data. The resampled three-dimensional (3D) face data are applied to extract novel descriptors including pyramidal shape index, pyramidal curvedness, pyramidal mean, and Gaussian curvatures. Proposed pyramidal shape maps are extracted at each level of the Gaussian pyramid on each point of the 3D data to have 2D matrices as representatives of 3D geometry information. A local descriptor structural context histogram, which represents the structure of the image using scale invariant feature transform, is calculated on pyramidal shape map descriptors and texture image to find matched keypoints in 3D and 2D modality, respectively. Score-level fusion by means of sum rule is employed to get a final matching score. Experimental results on the Face Recognition Grand Challenge (FRGC v2) database illustrate verification rates of 99 and 98.65% at 0.1% false acceptance rate for all versus all and ROC III experiments, respectively. On Bosphorus database, verification rate of 95.8% for neutral versus all experiment has been achieved.


    1. 1)
      • 1. Zhao, W., Chellappa, R., Phillips, P.J., et al: ‘Face recognition: a literature survey’, ACM Comput. Surv., 2003, 35, (4), pp. 399458.
    2. 2)
      • 2. Hailing, Z., Ajmal, M., Lei, W.: ‘Recent advances on single modal and multimodal face recognition: a survey’, IEEE Trans. Hum.-Mach. Syst., 2014, 44, (6), pp. 701716.
    3. 3)
      • 3. Mian, A.S., Bennamoun, M., Owens, R.: ‘An efficient multimodal 2D–3D hybrid approach to automatic face recognition’, IEEE Trans. PAMI, 2007, 29, (11), pp. 19271943.
    4. 4)
      • 4. Bowyer, K.W., Chang, K., Flynn, P.: ‘A survey of approaches and challenges in 3d and multi-modal 3d + 2d face recognition’, Comput. Vis. Image Underst., 2006, 101, (1), pp. 115.
    5. 5)
      • 5. Liu, P., Wang, Y., Huang, D., et al: ‘Learning the spherical harmonic features for 3-D face recognition’, IEEE Trans. Image Process., 2013, 22, (3), pp. 914925.
    6. 6)
      • 6. Mohammadzade, H., Hatzinakos, D.: ‘Iterative closet normal point for 3d face recognition’, IEEE Trans. PAMI, 2013, 35, (2), pp. 381397.
    7. 7)
      • 7. Lei, Y., Bennamoun, M., Hayat, M., et al: ‘An efficient 3D face recognition approach using local geometrical signatures’, Pattern Recogn., 2014, 47, (2), pp. 509524.
    8. 8)
      • 8. Bennamoun, M., Guo, Y., Sohel, F.: ‘Feature selection for 2D and 3D face recognition’, in Bennamoun, Mohammed, Guo, Yulan, Sohel, Ferdous (Eds.): ‘Wiley Encyclopedia of Electrical and Electronics EngineeringWiley Online Library (2015), pp. 154.
    9. 9)
      • 9. Huang, D., Soltana, W.B., Ardabilian, M., et al: ‘Textured 3d face recognition using biological vision-based facial representation and optimized weighted sum fusion’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition Workshops, June 2011, pp. 18.
    10. 10)
      • 10. Elaiwat, S., Bennamoun, M., Boussaid, F., et al: ‘A curvelet-based approach for textured 3D face recognition’, Pattern Recogn., 2015, 48, (4), pp. 12351246.
    11. 11)
      • 11. Mian, A., Bennamoun, M., Owens, R.: ‘Keypoint detection and local feature matching for textured 3d face recognition’, Int. J. Comput. Vis., 2008, 79, (1), pp. 112.
    12. 12)
      • 12. Zhang, G., Wang, Y.: ‘Robust 3D face recognition based on resolution invariant features’, Pattern Recognit. Lett., 2011, 32, (7), pp. 10091019.
    13. 13)
      • 13. Al-Osaimi, F.R., Bennamoun, M., Mian, A.: ‘Spatially optimized data-level fusion of texture and shape for face recognition’, IEEE Trans. on Image Process., 2012, 21, (2), pp. 859872.
    14. 14)
      • 14. Huang, D., Ardabilian, M., Wang, Y., et al: ‘3-D face recognition using eLBP-based facial description and local feature hybrid matching’, IEEE Trans. Inf. Forensics Sec., 2012, 7, (5), pp. 15511565.
    15. 15)
      • 15. Lei, Y., Bennamoun, M., El-Sallam, A.A.: ‘An efficient 3D face recognition approach based on the fusion of novel local low-level features’, Pattern Recogn., 2013, 46, (1), pp. 2437.
    16. 16)
      • 16. Liu, W., Yang, Y.: ‘Structural context for object categorization’. Proc. of Pacific-Rim Conf. on Multimedia, , 2009, pp. 280291.
    17. 17)
      • 17. Lowe, D.G.: ‘Distinctive image features from scale invariant keypoints’, Int. J. Comput. Vis., 2004, 60, (2), pp. 91110.
    18. 18)
      • 18. Guo, Y., Bennamoun, M., Sohel, F., et al: ‘A comprehensive performance evaluation of 3D local feature descriptors’, Int. J. Comput. Vis., 2016, 116, (1), pp. 6689.
    19. 19)
      • 19. Bayramoglu, N., Alatan, A.A.: ‘Shape index SIFT: range image recognition using local features’. Proc. of IEEE Int. Conf. on Pattern Recognition, August 2010, pp. 352355.
    20. 20)
      • 20. Inan, T., Halici, U.: ‘3-D face recognition with local shape descriptors’, IEEE Trans. Inf. Forensics Sec., 2012, 7, (2), pp. 577587.
    21. 21)
      • 21. Huang, D., Zhang, G., Ardabilian, M., et al: ‘3d face recognition using distinctiveness enhanced facial representations and local feature hybrid matching’. Proc. of IEEE Int. Conf. on Biometrics: Theory Applications and Systems (BTAS), September 2010, pp. 17.
    22. 22)
      • 22. Phillips, P.J., Flynn, P.J., Scruggs, T., et al: ‘Overview of the face recognition grand challenge’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, June 2005, pp. 947954.
    23. 23)
      • 23. Savran, A., Alyüz, N., Dibeklioğlu, H., et al: ‘Bosphorus database for 3D face analysis’. Proc. of European Workshop on Biometrics and Identity Management, May 2008, pp. 4756.
    24. 24)
      • 24. Soltanpour, S., Wu, Q.M.J., Anvaripour, M.: ‘Multimodal 2D–3D face recognition using structural context and pyramidal shape index’. Proc. of IET Conf. on Imaging for Crime Prevention and Detection (ICDP), July 2015, pp. 16.
    25. 25)
      • 25. Erdogmus, N., Dugelay, J.L.: ‘3D assisted face recognition: dealing with expression variations’, IEEE Trans. Inf. Forensics Sec., 2014, 9, (5), pp. 826838.
    26. 26)
      • 26. Alyuz, N., Gokberk, B., Akarun, L.: ‘Regional registration for expression resistant 3-D face recognition’, IEEE Trans. Inf. Forensics Sec., 2010, 5, (3), pp. 425440.
    27. 27)
      • 27. Belongie, S., Malik, J., Puzicha, J.: ‘Shape matching and object recognition using shape context’, IEEE Trans. PAMI, 2002, 24, (4), pp. 509522.
    28. 28)
      • 28. Li, H., Huang, D., Lemaire, P., et al: ‘Expression robust 3D face recognition via mesh-based histograms of multiple order surface differential quantities’. Proc. of Int. Conf. on Image Processing, September 2011, pp. 30533056.
    29. 29)
      • 29. Koenderink, J.J.: ‘Solid shape’ (Ser. Artificial Intelligence, MIT Press, Cambridge, MA, 1990).
    30. 30)
      • 30. Jain, A.K., Nandakumar, K., Ross, A.: ‘Score normalization in multimodal biometric systems’, Pattern Recogn., 2005, 38, (12), pp. 22702285.
    31. 31)
      • 31. Li, H., Huang, D., Morvan, J.M., et al: ‘Towards 3d face recognition in the real: a registration-free approach using fine-grained matching of 3d keypoint descriptors’, Int. J. Comput. Vis., 2015, 113, (2), pp. 128142.

Related content

This is a required field
Please enter a valid email address