Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Strengths and weaknesses of deep learning models for face recognition against image degradations

© The Institution of Engineering and Technology
Received 17/05/2017, Accepted 07/09/2017, Revised 14/08/2017, Published 13/09/2017

Convolutional neural network (CNN) based approaches are the state of the art in various computer vision tasks including face recognition. Considerable research effort is currently being directed toward further improving CNNs by focusing on model architectures and training techniques. However, studies systematically exploring the strengths and weaknesses of existing deep models for face recognition are still relatively scarce. In this paper, we try to fill this gap and study the effects of different covariates on the verification performance of four recent CNN models using the Labelled Faces in the Wild dataset. Specifically, we investigate the influence of covariates related to image quality and model characteristics, and analyse their impact on the face verification performance of different deep CNN models. Based on comprehensive and rigorous experimentation, we identify the strengths and weaknesses of the deep learning models, and present key areas for potential future research. Our results indicate that high levels of noise, blur, missing pixels, and brightness have a detrimental effect on the verification performance of all models, whereas the impact of contrast changes and compression artefacts is limited. We find that the descriptor-computation strategy and colour information does not have a significant influence on performance.

Inspec keywords: brightness; computer vision; feedforward neural nets; image coding; data compression; face recognition; image colour analysis; learning (artificial intelligence)

Other keywords: compression artifacts; blur; model characteristics; labelled faces in the wild dataset; SqueezeNet; image degradations; occlusion; descriptor-computation strategy; deep convolutional neural network based approach; image quality; missing pixels; face verification performance; image brightness; VGG-Face; descriptor computation; model architectures; face recognition; computer vision tasks; CNN architecture; colour information; deep CNN models; contrast; noise; learning techniques; AlexNet; deep learning model strength; GoogLeNet; deep learning model weakness; Joint Photographic Experts Group compression

Subjects: Knowledge engineering techniques; Image and video coding; Neural computing techniques; Image recognition; Computer vision and image processing techniques

References

    1. 1)
      • 39. Sun, Y., Chen, Y., Wang, X., et al: ‘Deep learning face representation by joint identification-verification’. Advances in Neural Information Processing Systems, 2014, pp. 1988–1996.
    2. 2)
      • 8. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 770–778.
    3. 3)
      • 9. Szegedy, C., Liu, W., Jia, Y., et al: ‘Going deeper with convolutions’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 1–9.
    4. 4)
      • 21. Szegedy, C., Vanhoucke, V., Ioffe, S., et al: ‘Rethinking the inception architecture for computer vision’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 2818–2826.
    5. 5)
      • 36. Kingma, D.P., Ba, J.L.: ‘Adam: a method for stochastic optimization’. Int. Conf. Learning Representation, 2015.
    6. 6)
      • 4. Ren, S., He, K., Girshick, R., et al: ‘Faster R-CNN: towards real-time object detection with region proposal networks’. Advances in Neural Information Processing Systems, 2015, pp. 91–99.
    7. 7)
      • 29. Dosovitskiy, A., Springenberg, J.T., Riedmiller, M., et al: ‘Discriminative unsupervised feature learning with convolutional neural networks’. Advances in Neural Information Processing Systems, 2014, pp. 766–774.
    8. 8)
      • 26. Dodge, S., Karam, L.: ‘Understanding how image quality affects deep neural networks’. 2016 Eighth Int. Conf. Quality of Multimedia Experience (QoMEX), 2016, pp. 1–6.
    9. 9)
      • 15. Sharma, A., Tuzel, O., Jacobs, D.W.: ‘Deep hierarchical parsing for semantic segmentation’. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 530–538.
    10. 10)
      • 13. Badrinarayanan, V., Handa, A., Cipolla, R.: ‘SegNet: a deep convolutional encoder–decoder architecture for robust semantic pixel-wise labelling’, arXiv preprint arXiv:150507293, 2015.
    11. 11)
      • 38. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778.
    12. 12)
      • 30. Zeiler, M.D., Fergus, R.: ‘Visualizing and understanding convolutional networks’. European Conf. Computer Vision – ECCV, 2014, pp. 818–833.
    13. 13)
      • 20. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘ImageNet classification with deep convolutional neural networks’. Advances in Neural Information Processing Systems, 2012, pp. 1097–1105.
    14. 14)
      • 3. Gidaris, S., Komodakis, N.: ‘Object detection via a multi-region and semantic segmentation-aware CNN model’. Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 1134–1142.
    15. 15)
      • 31. Lenc, K., Vedaldi, A.: ‘Understanding image representations by measuring their equivariance and equivalence’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 991–999.
    16. 16)
      • 34. Chaib, S., Yao, H., Gu, Y., et al: ‘Deep feature extraction and combination for remote sensing image classification based on pre-trained CNN models’. Ninth Int. Conf. Digital Image Processing (ICDIP 2017), 2017, pp. 104203D–104203D.
    17. 17)
      • 23. Jain, A.K., Li, S.Z.: ‘Handbook of face recognition’ (Springer, 2011).
    18. 18)
      • 18. Schroff, F., Kalenichenko, D., Philbin, J.: ‘FaceNet: a unified embedding for face recognition and clustering’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 815–823.
    19. 19)
      • 17. Taigman, Y., Yang, M., Ranzato, M., et al: ‘Deepface: closing the gap to human-level performance in face verification’. CVPR, 2014, pp. 1701–1708.
    20. 20)
      • 11. Wang, N., Yeung, D.Y.: ‘Learning a deep compact image representation for visual tracking’. Advances in neural information processing systems, 2013, pp. 809–817.
    21. 21)
      • 2. Li, G., Yu, Y.: ‘Deep contrast learning for salient object detection’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016.
    22. 22)
      • 24. Ghazi, M.M., Ekenel, H.K.: ‘A comprehensive analysis of deep learning based representation for face recognition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2016, pp. 34–41.
    23. 23)
      • 10. Alahi, A., Goel, K., Ramanathan, V., et al: ‘Social LSTM: human trajectory prediction in crowded spaces’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 961–971.
    24. 24)
      • 6. Gidaris, S., Komodakis, N.: ‘LocNet: improving localization accuracy for object detection’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 789–798.
    25. 25)
      • 7. Liu, N., Han, J.: ‘DHSNet: deep hierarchical saliency network for salient object detection’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 678–686.
    26. 26)
      • 22. Iandola, F.N., Han, S., Moskewicz, M.W., et al: ‘SqueezeNet: AlexNet-level accuracy with 50× fewer parameters and <0.5 mb model size’, arXiv preprint arXiv:160207360, 2016.
    27. 27)
      • 32. Huang, G.B., Miller, E.L.: ‘Labeled faces in the wild: updates and new reporting procedures’. Technical Report, UM-CS-2014-003, 2014.
    28. 28)
      • 19. Parkhi, O.M., Vedaldi, A., Zisserman, A.: ‘Deep face recognition’. BMVC, 2015, p. 6.
    29. 29)
      • 12. Wang, L., Ouyang, W., Wang, X., et al: ‘STCT: sequentially training convolutional networks for visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 1373–1381.
    30. 30)
      • 37. Kazemi, V., Sullivan, J.: ‘One millisecond face alignment with an ensemble of regression trees’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2014, pp. 1867–1874.
    31. 31)
      • 27. Chatfield, K., Simonyan, K., Vedaldi, A., et al: ‘Return of the devil in the details: delving deep into convolutional nets’, arXiv preprint arXiv:14053531, 2014.
    32. 32)
      • 33. Razavian, A.S., Azizpour, H., Sullivan, J., et al: ‘CNN features off-the-shelf: an astounding baseline for recognition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, 2014, pp. 806–813.
    33. 33)
      • 35. Russakovsky, O., Deng, J., Su, H., et al: ‘ImageNet large scale visual recognition challenge’, Int. J. Comput. Vis., 2015, 115, (3), pp. 211–252.
    34. 34)
      • 16. Huang, G.B., Ramesh, M., Berg, T., et al: ‘Labeled faces in the wild: a database for studying face recognition in unconstrained environments’. Technical Report, 07-49, University of Massachusetts, Amherst, 2007.
    35. 35)
      • 14. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Semantic image segmentation with deep convolutional nets and fully connected CRFs’, arXiv preprint arXiv:14127062, 2014.
    36. 36)
      • 40. Chen, J.C., Sankaranarayanan, S., Patel, V.M., et al: ‘Unconstrained face verification using Fisher vectors computed from frontalized faces’. 2015 IEEE 7th Int. Conf. Biometrics Theory, Applications and Systems (BTAS), 2015, pp. 1–8.
    37. 37)
      • 28. Richard-Webster, B., Anthony, S.E., Scheirer, W.J.: ‘Psyphy: a psychophysics driven evaluation framework for visual recognition’, arXiv preprint arXiv:161106448, 2016.
    38. 38)
      • 1. Bruce, N.D., Catton, C., Janjic, S.: ‘A deeper look at saliency: feature contrast, semantics, and beyond’. Proc. the IEEE Conf. on Computer Vision and Pattern Recognition, 2016, pp. 516–524.
    39. 39)
      • 5. Girshick, R., Donahue, J., Darrell, T., et al: ‘Rich feature hierarchies for accurate object detection and semantic segmentation’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2014, pp. 580–587.
    40. 40)
      • 25. Karahan, S., Yildirum, M.K., Kirtac, K., et al: ‘How image degradations affect deep CNN-based face recognition?’. 2016 Int. Conf. Biometrics Special Interest Group (BIOSIG), 2016, pp. 1–5.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-bmt.2017.0083
Loading

Related content

content/journals/10.1049/iet-bmt.2017.0083
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address