Animal classification using facial images with score-level fusion

Animal classification using facial images with score-level fusion

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Computer Vision — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

A real-world animal biometric system that detects and describes animal life in image and video data is an emerging subject in machine vision. These systems develop computer vision approaches for the classification of animals. A novel method for animal face classification based on score-level fusion of recently popular convolutional neural network (CNN) features and appearance-based descriptor features is presented. This method utilises a score-level fusion of two different approaches; one uses CNN which can automatically extract features, learn and classify them; and the other one uses kernel Fisher analysis (KFA) for its feature extraction phase. The proposed method may also be used in other areas of image classification and object recognition. The experimental results show that automatic feature extraction in CNN is better than other simple feature extraction techniques (both local- and appearance-based features), and additionally, appropriate score-level combination of CNN and simple features can achieve even higher accuracy than applying CNN alone. The authors showed that the score-level fusion of CNN extracted features and appearance-based KFA method have a positive effect on classification accuracy. The proposed method achieves 95.31% classification rate on animal faces which is significantly better than the other state-of-the-art methods.


    1. 1)
      • 1. Elson, J., Douceur, J., Howell, J., et al: ‘Asirra: a CAPTCHA that exploits interest-aligned manual image categorization’. Proc. ACM Conf. Computer and Communications Security (CCS), Alexandria VA, USA, 2007, pp. 366374.
    2. 2)
      • 2. Marcialis, G., Roli, F.: ‘Score-level fusion of fingerprint and face matchers for personal verification under stress conditions’. 14th IEEE Int. Conf. Image Analysis and Processing ICIAP, DC, USA, 2007, pp. 259264.
    3. 3)
      • 3. Elmir, Y., Elberrichi, Z., Adjoudj, R.: ‘Score-level fusion based multimodal biometric identification (fingerprint & voice)’. 6th Int. Conf. Sciences of Electronics, Technologies of Information and Telecommunications, Sousse, 2010, pp. 146150.
    4. 4)
      • 4. Sim, H., Hishammuddin, A., Rohayanti, H., et al: ‘Multimodal biometrics: weighted score-level fusion based on non-ideal iris and face images’, Expert Syst. Appl., 2014, 41, (11), pp. 53905404.
    5. 5)
      • 5. Patil, A., Bhalke, D.: ‘Fusion of fingerprint, palmprint and iris for person identification’. Int. Conf. Automatic Control and Dynamic Optimization Techniques (ICACDOT), Pune, 2016, pp. 960963.
    6. 6)
      • 6. Takimoto, H., Mitsukura, Y., Fukumi, M., et al: ‘Robust gender and age estimation under varying facial pose’, Electron. Commun. Jpn., 2008, 91, (7), pp. 3240.
    7. 7)
      • 7. Schmid, C.: ‘Constructing models for content-based image retrieval’. Proc. 2001 IEEE Computer Society Conf. Computer Vision and Pattern Recognition, 2001. CVPR 2001, Kauai, USA, December 2001, pp. 1139.
    8. 8)
      • 8. Ramanan, D., Forsyth, D.A., Barnard, K.: ‘Detecting, localizing and recovering kinematics of textured animals’. 2005 IEEE Computer Society Conf. Computer Vision and Pattern Recognition, San Diego, USA, June 2005, pp. 635642.
    9. 9)
      • 9. Ramanan, D., Forsyth, D.A., Barnard, M.-K.: ‘Building models of animals from video’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (8), pp. 13191334.
    10. 10)
      • 10. Berg, T.L., Forsyth, D.A.: ‘Animals on the web’. 2006 IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR'06), NY, USA, 2006, pp. 14631470.
    11. 11)
      • 11. Penga, Z., Lia, Y., Caib, Z., et al: ‘Deep boosting: joint feature selection and analysis dictionary learning in hierarchy’, Neurocomputing, 2016, 178, (20), pp. 3645.
    12. 12)
      • 12. Afkham, H., Tavakoli, A., Eklundh, J., et al: ‘Joint visual vocabulary for animal classification’. Int. Conf. Pattern Recognition, 2008. ICPR 2008, Tampa, FL, USA, 2008, pp. 14.
    13. 13)
      • 13. Si, Z., Zhu, S.-C.: ‘Learning hybrid image templates (HIT) by information projection’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (7), pp. 13541367.
    14. 14)
      • 14. Druzhkov, P.N., Kustikova, V.D.: ‘A survey of deep learning methods and software tools for image classification and object detection’, Pattern Recognit. Image Anal., 2016, 26, (1), pp. 915.
    15. 15)
      • 15. Krizhevsky, A., Sutskever, I., Hinton, G.: ‘Imagenet classification with deep convolutional neural networks’. Advances in Neural Information Processing Systems, Lake Tahoe, USA, 2012, pp. 10971105.
    16. 16)
      • 16. Russakovsky, O., Deng, J., Su, H., et al: ‘Imagenet large scale visual recognition challenge’, Int. J. Comput. Vision, 2015, 115, (3), pp. 211252.
    17. 17)
      • 17. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:1409.1556.
    18. 18)
      • 18. Deng, J., Dong, W., Socher, R.: ‘Imagenet: a large-scale hierarchical image database’. Proc. IEEE Conf. Computer Vision Pattern Recognition, Miami FL, USA, 2009, pp. 248255.
    19. 19)
      • 19. Dalal, N., Triggs, B.: ‘Histograms of oriented gradients for human detection’. Computer Vision and Pattern Recognition, San Diego, CA, USA, June 2005, pp. 886893.
    20. 20)
      • 20. Farmanbar, M., Toygar, Ö.: ‘Feature selection for the fusion of face and palmprint biometrics’, Signal Image Video Process., 2016, 10, (5), pp. 951958.
    21. 21)
      • 21. Eskandari, M., Toygar, Ö.: ‘Fusion of face and iris biometrics using local and global feature extraction methods’, Signal Image Video Process., 2014, 8, (6), pp. 9951006.
    22. 22)
      • 22. Ojala, T., Pietikäinen, M., Maenpää, T.: ‘Multiresolution gray-scale and rotation invariant texture classification with local binary patterns’, IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24, (7), pp. 971987.
    23. 23)
      • 23. Liu, L., Lao, S., Fieguth, P., et al: ‘Median robust extended local binary pattern for texture classification’, IEEE Trans. Image Process, 2016, 25, (3), pp. 13681381.
    24. 24)
      • 24. Liu, L., Fieguth, P., Guo, Y., et al: ‘Local binary features for texture classification: taxonomy and experimental study’, Pattern Recognit., 2017, 62, pp. 135160.
    25. 25)
      • 25. Liu, C.: ‘Capitalize on dimensionality increasing techniques for improving face recognition grand challenge performance’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (5), pp. 725737.
    26. 26)
      • 26. Riesenhuber, M., Poggio, T.: ‘Hierarchical models of object recognition in cortex’, Nat. Neurosci., 1999, 2, pp. 10191025.
    27. 27)
      • 27. Cord, M., Theriault, C., Thome, N.: ‘HMAX-S: deep scale representation for biologically inspired image categorization’. Proc. IEEE Int. Conf. Image Processing, Brussels, Belgium, 2011, pp. 12611264.
    28. 28)
      • 28. Thériault, C., Thome, N., Cord, M.: ‘Extended coding and pooling in the HMAX model’, IEEE Trans. Image Process., 2013, 22, (2), pp. 764777.
    29. 29)
      • 29. Bingpeng, M., Su, Y., Jurie, F.: ‘Covariance descriptor based on bio-inspired features for person re-identification and face verification’, Image Vision Comput., 2014, 32, (6-7), pp. 379390.
    30. 30)
      • 30. Belhumeur, P.N., Hespanha, J.P., Kriegman, D.J.: ‘Eigenfaces vs. fisherfaces: recognition using cla ss specific linear projection’, IEEE Trans. Pattern Anal. Mach. Intell., 1997, 19, pp. 711720.
    31. 31)
      • 31. Eskandari, M., Toygar, Ö.: ‘Selection of optimized features and weights on face-iris fusion using distance images’, Comput. Vis. Image Underst., 2015, 137, pp. 6375.
    32. 32)
      • 32. Si, Z., Zhu, S.-C.: ‘Learning and-or templates for object recognition and detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (9), pp. 21892205.
    33. 33)
      • 33. Guo, Z., Zhang, D., Zhang, D.: ‘A completed modeling of local binary pattern operator for texture classification’, IEEE Trans. Image Process., 2010, 19, (6), pp. 16571663.
    34. 34)
      • 34. Ahonen, T., Matas, J., He, C., et al: ‘Rotation invariant image description with local binary pattern histogram Fourier features’, Lect. Notes Comput. Sci., 2009, 5575, pp. 6170.
    35. 35)
      • 35. Donahue, J., Jia, Y., Vinyals, O., et al: ‘DeCAF: a deep convolutional activation feature for generic visual recognition’. Int. Conf. Machine Learning (ICML), Beijing, China, 2014, pp. 647655.
    36. 36)
      • 36. Nanni, L., Ghidoni, S., Brahnamb, S.: ‘Handcrafted vs. non-handcrafted features for computer vision classification’, Pattern Recognit., 2017, 71, pp. 158172.
    37. 37)
      • 37. He, M., Horng, S., Fan, F., et al: ‘Performance evaluation of score level fusion in multimodal’, Biometric Syst. Pattern Recognit., 2010, 43, pp. 17891800.
    38. 38)
      • 38. Felzenszwalb, P., Girshick, R., McAllester, D., et al: ‘Object detection with discriminatively trained part-based models’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32, (9), pp. 16271645.

Related content

This is a required field
Please enter a valid email address