Accurate gathering of phenotypic information is a key aspect in several subject matters, including biometrics, biomedical analysis, forensics, and many other. Automatic identification of anatomical structures of biometric interest, such as fingerprints, iris patterns, or facial traits, are extensively used in applications like access control and anthropological research, all having in common the drawback of requiring intrusive means for acquiring the required information. In this regard, the ear structure has multiple advantages. Not only the ear's biometric markers can be easily captured from the distance with non intrusive methods, but also they experiment almost no changes over time, and are not influenced by facial expressions. Here we present a new method based on Geometric Morphometrics and Deep Learning for automatic ear detection and feature extraction in the form of landmarks. A convolutional neural network was trained with a set of manually landmarked examples. The network is able to provide morphometric landmarks on ears' images automatically, with a performance that matches human landmarking. The feasibility of using ear landmarks as feature vectors opens a novel spectrum of biometrics applications.

References

1. 1)
  - 50. Geurts, P., Ernst, D., Wehenkel, L.: ‘Extremely randomized trees’, Mach. Learn., 2006, 63, (1), pp. 3–42.
2. 2)
  - 24. Kumar, A., Chan, T.-S.T.: ‘Robust ear identification using sparse representation of local texture descriptors’, Pattern Recogn., 2013, 46, (1), pp. 73–85.
3. 3)
  - 32. Goodall, C.: ‘Procrustes methods in the statistical analysis of shape’, J. R. Stat. Soc. B (Methodological), 1991, 53, pp. 285–339. http://www.jstor.org/stable/2345744.
4. 4)
  - 14. Prakash, S., Gupta, P.: ‘An efficient ear localization technique’, Image Vis. Comput., 2012, 30, (1), pp. 38–50.
5. 5)
  - 39. LeCun, Y., Bottou, L., Bengio, Y., et al: ‘Gradient-based learning applied to document recognition’, Proc. IEEE, 1998, 86, (11), pp. 2278–2323.
6. 6)
  - 3. Gómez-Valdés, J.A., Hünemeier, T., Contini, V., et al: ‘Fibroblast growth factor receptor 1 (FGFR1) variants and craniofacial variation in Amerindians and related populations’, Am. J. Human Biol, 2013, 25, pp. 12–19. http://www.ncbi.nlm.nih.gov/pubmed/23070782.
7. 7)
  - 43. Ruiz-Linares, A., Adhikari, K., Acuña-Alonzo, V., et al: ‘Admixture in Latin America: geographic structure, phenotypic diversity and self-perception of ancestry based on 7,342 individuals’, PLoS Genet., 2014, 10, (9), p. e1004572. http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1004572.
8. 8)
  - 45. Dieleman, S., Schlüter, J., Raffel, C., et al: ‘Lasagne: first release’, August 2015. http://dx.doi.org/10.5281/zenodo.27878.
9. 9)
  - 10. Chen, H., Bhanu, B.: ‘Shape model-based 3D ear detection from side face range images’. 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR'05) – Workshops. IEEE, vol. 3, pp. 122–122. http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=1565434.
10. 10)
  - 42. Boureau, Y.-L., Ponce, J., Lecun, Y.: ‘A theoretical analysis of feature pooling in visual recognition’. Proc. of the 27th International Conf. on Machine Learning (2010), 2010, pp. 111–118. http://www.ece.duke.edu/{∼}lcarin/icml2010b.pdf.
11. 11)
  - 2. Liu, F., van der Lijn, F., Schurmann, C., et al: ‘A Genome-Wide Association Study Identifies Five Loci Influencing Facial Morphology in Europeans’, PLoS Genet., 2012, 8, (9): e1002932. doi: 10.1371/journal.pgen.1002932.
12. 12)
  - 9. Iannarelli, A.V.: ‘Ear identification’ (Paramont Publishing Company, 1989), vol. 1. https://books.google.com/books?id=jgPkAAAACAAJ{&}pgis=1.
13. 13)
  - 7. Zelditch, M.L., Swiderski, D.L., Sheets, H.D., et al: ‘Geometric morphometrics for biologists’, Elsevier, 2004, 59, (3), p. 457. http://www.sciencedirect.com/science/article/B848M-4MWYDVJ-7/2/2ec7a8306777645c5716a53774ec699c$\delimiter‘026E30F$nhttp://books.google.com/books?id=LKCVAGn8vkoC{&}pgis=1.
14. 14)
  - 11. Chen, H., Bhanu, B.: ‘Contour matching for 3D ear recognition’. Seventh IEEE Workshop on Applications of Computer Vision, WACV 2005, 2007, pp. 123–128.
15. 15)
  - 31. Adhikari, K., Reales, G., Smith, A.J.P., et al: ‘A genome-wide association study identifies multiple loci for variation in human ear morphology’, Nat. Commun., 2015, 6, (May), p. 7500. http://www.nature.com/doifinder/10.1038/ncomms8500.
16. 16)
  - 16. Purkait, R., Singh, P.: ‘A test of individuality of human external ear pattern: its application in the field of personal identification’, Forensic Sci. Int., 2008, 178, (2–3), pp. 112–118.
17. 17)
  - 8. Pflug, S., Busch, C.: ‘Ear biometrics: a survey of detection, feature extraction and recognition methods’, IET Biometrics, 2012, 1, (2), p. 114.
18. 18)
  - 22. Islam, S.M.S., Bennamoun, M., Davies, R.: ‘Fast and fully automatic ear detection using cascaded adaboost’. 2008 IEEE Workshop on Applications of Computer Vision, WACV, 2008.
19. 19)
  - 35. Taigman, Y., Yang, M., Ranzato, M., et al: ‘DeepFace: closing the gap to human-level performance in face verification’. Conf. on Computer Vision and Pattern Recognition (CVPR), 2014, p. 8. http://www.cs.tau.ac.il/{∼}wolf/papers/deepface{_}11{_}01{_}2013.pdf.
20. 20)
  - 12. Attarchi, S., Faez, K., Rafiei, A.: Advanced Concepts for Intelligent Vision Systems, October 2008 (LNCS, 5259). http://dl.acm.org/citation.cfm?id=1462298.1462399.
21. 21)
  - 49. Solina, F., Peer, P., Batagelj, B., et al: ‘Color-based face detection in the’ 15 seconds of fame’ art installation’. Proc. of Mirage 2003, Conf. on Computer Vision/Computer Graphics, 2003, pp. 38–47.
22. 22)
  - 21. Abaza, A., Hebert, C., Harrison, M.A.F.: ‘Fast learning ear detection for real-time surveillance’. IEEE 4th International Conf. on Biometrics: Theory, Applications and Systems, BTAS 2010, 2010.
23. 23)
  - 34. Ma, J., Sheridan, R.P., Liaw, A., et al: ‘Deep neural nets as a method for quantitative structureactivity relationships’, J. Chem. Inf. Model., 2015, 55, (2), pp. 263–274. http://www.ncbi.nlm.nih.gov/pubmed/25635324.
24. 24)
  - 26. Chan, T.-S., Kumar, A.: ‘Reliable ear identification using 2-d quadrature filters’, Pattern Recognit. Lett., 2012, 33, (14), pp. 1870–1881.
25. 25)
  - 27. Mitteroecker, P., Gunz, P.: Evol Biol, 2009, 36: 235. doi:10.1007/s11692-009-9055-x.
26. 26)
  - 33. Ciodaro, T., Deva, D., de Seixas, J.M., et al: ‘Online particle detection with neural networks based on topological calorimetry information’, J. Phys. Conf. Ser., 2012, 368, (1), p. 012030. http://iopscience.iop.org/article/10.1088/1742-6596/368/1/012030.
27. 27)
  - 30. Alexander, K.S., Stott, D.J., Sivakumar, B., et al: ‘A morphometric study of the human ear’, J. Plastic Reconstruct. Aesthetic Surgery, 2011, 64, (1), pp. 41–47. http://www.ncbi.nlm.nih.gov/pubmed/20447883.
28. 28)
  - 20. Sibai, F.N., Nuaimi, A., Maamari, A., et al: ‘Ear recognition with feed-forward artificial neural networks’, Neural Comput. Appl., 2013, 23, (5), pp. 1265–1273. http://link.springer.com/10.1007/s00521-012-1068-1.
29. 29)
  - 53. Cox, T.C., Camci, E.D., Vora, S., et al: ‘The genetics of auricular development and malformation: new findings in model systems driving future directions for microtia research’, Eur. J. Med. Genet., 2014, 57, (8), pp. 394–401. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=4143470{&}tool=pmcentrez{&}rendertype=abstract.
30. 30)
  - 46. Hinton, G.E., Srivastava, N., Krizhevsky, A., et al: ‘Improving neural networks by preventing co-adaptation of feature detectors’, arXiv: 1207.0580, 2012, pp. 1–18. http://arxiv.org/abs/1207.0580.
31. 31)
  - 1. Paternoster, L., Zhurov, A.I., Toma, A.M., et al: ‘Genome-wide association study of three-dimensional facial morphology identifies a variant in PAX3 associated with nasion position’, Am. J. Hum. Genet., 2012, 90, pp. 478–485.
32. 32)
  - 52. Faris, C.: ‘Scott-Brown's otorhinolaryngology, head and neck surgery, 7th edn’, October 2011, p. 559. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3604940/.
33. 33)
  - 6. Ibrahim, M.I.S., Nixon, M.S., Mahmoodi, S.: ‘The effect of time on ear biometrics’. 2011 International Joint Conf. on Biometrics (IJCB). IEEE, October 2011, pp. 1–6. http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=6117584.
34. 34)
  - 51. Wehenkel, L., Ernst, D., Geurts, P.: ‘Ensembles of extremely randomized trees and some generic applications’. Robust Methods for Power System State Estimation and Load Forecasting, 2006.
35. 35)
  - 18. Pflug, A., Winterstein, A., Busch, C.: ‘Robust localization of ears by feature level fusion and context information’. Biometrics (ICB), 2013 International Conf. on. IEEE, 2013, pp. 1–8.
36. 36)
  - 19. Liu, Y., Zhang, B., Zhang, D.: ‘Ear-parotic face angle: a unique feature for 3D ear recognition’, Pattern Recognit. Lett., 2015, 53, pp. 9–15. http://linkinghub.elsevier.com/retrieve/pii/S0167865514003316.
37. 37)
  - 4. Alberink, I., Ruifrok, A.: ‘Performance of the FearID earprint identification system’, Forensic Sci. Int., 2007, 166, (2–3), pp. 145–154. http://www.ncbi.nlm.nih.gov/pubmed/16772109/23/11/16.
38. 38)
  - 29. Azaria, R., Adler, N., Silfen, R., et al: ‘Morphometry of the adult human earlobe: a study of 547 subjects and clinical application’, Plast. Reconstr. Surg., 2003, 111, (7), pp. 2398–2402; discussion 2403-4. http://www.ncbi.nlm.nih.gov/pubmed/12794488.
39. 39)
  - 37. Dieleman, S., Willett, K.W., Dambre, J.: ‘Rotation-invariant convolutional neural networks for galaxy morphology prediction’, Mon. Not. R. Astron. Soc., 2015, 450, (2), pp. 1441–1459. http://arxiv.org/abs/1503.07077.
40. 40)
  - 15. Yan, P., Bowyer, K.W.: ‘Biometric recognition using 3D ear shape’, IEEE Trans. Pattern Anal. Mach. Intell., 2007, 29, (8), pp. 1297–1308. http://www.ncbi.nlm.nih.gov/pubmed/17568136.
41. 41)
  - 23. Yuan, L., Liu, W., Li, Y.: ‘Non-negative dictionary based sparse representation classification for ear recognition with occlusion’, Neurocomputing, 2016, 171, pp. 540–550.
42. 42)
  - 36. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, (7553), pp. 436–444. http://dx.doi.org/10.1038/nature14539$\delimiter‘026E30F$n10.1038/nature14539.
43. 43)
  - 44. Viola, P., Jones, M.: ‘Robust real-time object detection’, Int. J. Comput. Vis., 2001, 57, pp. 137–154. http://scholar.google.com/scholar?hl=en{&}btnG=Search{&}q=intitle:Robust+Real-time+Object+Detection{#}0.
44. 44)
  - 40. Toshev, A., Szegedy, C.: ‘DeepPose: human pose estimation via deep neural networks’. Computer Vision and Pattern Recognition (CVPR), 2014, pp. 1653–1660. http://arxiv.org/abs/1312.4659.
45. 45)
  - 25. Kumar, A., Wu, C.: ‘Automated human identification using ear imaging’, Pattern Recogn., 2012, 45, (3), pp. 956–968.
46. 46)
  - 47. Çeliktutan, O., Ulukaya, S., Sankur, B., et al: ‘A comparative study of face landmarking techniques’, EURASIP J. Image and Video Process., 2013, 2013, (1), p. 13. http://jivp.eurasipjournals.springeropen.com/articles/10.1186/1687-5281-2013-13.
47. 47)
  - 41. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘ImageNet classification with deep convolutional neural networks’. Advances in Neural Information Processing Systems, 2012, pp. 1–9.
48. 48)
  - 28. Slice, D.: ‘Modern morphometrics’. Modern morphometrics in physical anthropology, 2005, pp. 1–46. http://link.springer.com/chapter/10.1007/0-387-27614-9{_}1.
49. 49)
  - 48. Pedregosa, F., Varoquaux, G., Gramfort, A., et al: ‘Scikit-learn: machine learning in Python’, J. Mach. Learn. Res., 2011, 12, pp. 2825–2830.
50. 50)
  - 38. Fukushima, K.: ‘Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position’, Biol. Cybern., 1980, 36, (4), pp. 193–202.
51. 51)
  - 13. Ansari, S., Gupta, P.: ‘Localization of ear using outer helix curve of the ear’. International Conf. on Computing: the Theory and Applications, 2007, pp. 688–692.
52. 52)
  - 5. Sforza, C., Grandi, G., Binelli, M., et al: ‘Age- and sex-related changes in the normal human ear’, Forensic Sci. Int., 2009, 187, (1–3), p. 110.e1–7. http://www.ncbi.nlm.nih.gov/pubmed/19356871.
53. 53)
  - 17. Ercan, I., Ozdemir, S.T., Etoz, A., et al: ‘Facial asymmetry in young healthy subjects evaluated by statistical shape analysis’, J. Anat., 2008, 213, (6), pp. 663–669.

Automatic ear detection and feature extraction using Geometric Morphometrics and convolutional neural networks

References

Related content