Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Real-time AdaBoost cascade face tracker based on likelihood map and optical flow

The authors present a novel face tracking approach where optical flow information is incorporated into a modified version of the Viola–Jones detection algorithm. In the original algorithm, detection is static, as information from previous frames is not considered; in addition, candidate windows have to pass all stages of the classification cascade, otherwise they are discarded as containing no face. In contrast, the proposed tracker preserves information about the number of classification stages passed by each window. Such information is used to build a likelihood map, which represents the probability of having a face located at that position. Tracking capabilities are provided by extrapolating the position of the likelihood map to the next frame by optical flow computation. The proposed algorithm works in real time on a standard laptop. The system is verified on the Boston Head Tracking Database, showing that the proposed algorithm outperforms the standard Viola–Jones detector in terms of detection rate and stability of the output bounding box, as well as including the capability to deal with occlusions. The authors also evaluate two recently published face detectors based on convolutional networks and deformable part models with their algorithm showing a comparable accuracy at a fraction of the computation time.

References

    1. 1)
      • 72. Yan, J., Zhang, X., Lei, Z., et al: ‘Face detection by structural models’, Image Vis. Comput., 2014, 32, (10), pp. 790799, Best of Automatic Face and Gesture Recognition 2013.
    2. 2)
      • 46. Yun, T., Guan, L.: ‘Fiducial point tracking for facial expression using multiple particle filters with kernel correlation analysis’. Proc. IEEE Int. Conf. Image Processing, ICIP, September 2010, pp. 373376.
    3. 3)
      • 2. Jain, A.K., Kumar, A.: ‘Biometrics of next generation: an overview’, in Mordini, E., Tzovaras, D. (Eds.): ‘Second generation biometrics’ (Springer, 2010).
    4. 4)
      • 13. Li, Y.M., Gong, S.G., Liddell, H.: ‘Support vector regression and classification based multi-view face detection and recognition’. Proc. Int. Conf. Automatic Face and Gesture Recognition, FG, 2000.
    5. 5)
      • 63. Chen, W., Zhang, K., Liu, Q.: ‘Robust visual tracking via patch based kernel correlation filters with adaptive multiple feature ensemble’, Neurocomputing, 2016, 214, pp. 607617.
    6. 6)
      • 41. Nguyen, Q.-A., Robles-Kelly, A., Shen, C.: ‘Enhanced kernel-based tracking for monochromatic and thermographic video’. IEEE Int. Conf. Video and Signal Based Surveillance, AVSS, November 2006, pp. 2828.
    7. 7)
      • 29. Dornaika, F., Ahlberg, J.: ‘Fitting 3d face models for tracking and active appearance model training’, Image Vis. Comput., 2006, 24, (9), pp. 10101024.
    8. 8)
      • 27. Zhao, W., Chellappa, R., Phillips, P., et al: ‘Face recognition: a literature survey’, ACM Comput. Surv., 2003, 35, (4), pp. 399458.
    9. 9)
      • 48. Zhu, Z., Ji, Q.: ‘3d face pose tracking from an uncalibrated monocular camera’. Proc. Int. Conf. Pattern Recognition, ICPR, August 2004, vol. 4, pp. 400403.
    10. 10)
      • 77. Duong, C.N., Dinh, T.C.P., Ngo, T.D., et al: ‘Robust eye localization in video by combining eye detector and eye tracker’. Proc. Int. Conf. Pattern Recognition, ICPR, November 2012, pp. 242245.
    11. 11)
      • 74. La Cascia, M., Sclaroff, S., Athitsos, V.: ‘Fast, reliable head tracking under varying illumination: an approach based on registration of texture-mapped 3d models’, IEEE Trans. Pattern Anal. Mach. Intell., 22, (4), 2000, pp. 322336.
    12. 12)
      • 64. Bigun, J.: ‘Vision with direction’ (Springer, 2006).
    13. 13)
      • 4. Viola, P., Jones, M.: ‘Rapid object detection using a boosted cascade of simple features’. Proc. Computer Vision and Pattern Recognition Conf., CVPR, December 2001, vol. 1, pp. 511518.
    14. 14)
      • 60. Zhou, L.B., Wang, H., Mou, W., et al: ‘Robust face detection and tracking under natural conditions’. Proc. Int. Conf. Robotics and Biomimetics, ROBIO, December 2013, pp. 934939.
    15. 15)
      • 42. Vadakkepat, P., Lim, P., De-Silva, L.C., et al: ‘Multimodal approach to human-face detection and tracking’, IEEE Trans. Ind. Electron., 2008, 55, (3), pp. 13851393.
    16. 16)
      • 49. Girondel, V., Caplier, A., Bonnaud, L.: ‘Real time tracking of multiple persons by Kalman filtering and face pursuit for multimedia applications’. Proc. IEEE Southwest Symp. Image Analysis and Interpretation, March 2004, pp. 201205.
    17. 17)
      • 47. Chae, Y.N., Ha, J., Yang, H.S.: ‘Development of an efficient face detection and tracking system for mobile devices’. Proc. IEEE Int. Conf. Virtual Systems and Multimedia, VSMM, October 2010, pp. 192196.
    18. 18)
      • 37. Kalal, Z., Mikolajczyk, K., Matas, J.: ‘Face-TLD: tracking-learning-detection applied to faces’. Proc. Int. Conf. Image Processing, ICIP, 2010, pp. 37893792.
    19. 19)
      • 73. Zhu, X., Ramanan, D.: ‘Face detection, pose estimation, and landmark localization in the wild’. Proc. Conf. Computer Vision and Pattern Recognition, CVPR, June 2012, pp. 28792886.
    20. 20)
      • 68. Lucas, B.D., Kanade, T.: ‘An iterative image registration technique with an application to stereo vision’. Proc. Int. Joint Conf. Artificial Intelligence, IJCAI, October 1981, pp. 674679.
    21. 21)
      • 55. Subban, R., Muthukumar, S., Pasupathi, P., et al: ‘Face tracking techniques in color images: a study and review’, Int. J. Eng. Res. Technol., 2013, 2, (12), pp. 24812487.
    22. 22)
      • 54. Wu, B., Hu, B.-G., Ji, Q.: ‘A coupled hidden markov random field model for simultaneous face clustering and tracking in videos’, Pattern Recognit., 2017, 64, pp. 361373.
    23. 23)
      • 59. My, V.D., Zell, A.: ‘Real time face tracking and pose estimation using an adaptive correlation filter for human-robot interaction’. Proc. European Conf. Mobile Robots, September 2013, pp. 119124.
    24. 24)
      • 67. Harris, C., Stephens, M.: ‘A combined corner and edge detector’. Proc. Fourth Alvey Vision Conf., 1988, pp. 147151.
    25. 25)
      • 17. Zhang, L., Chu, R., Xiang, S., et al: ‘Face detection based on multi-block lbp representation’. Proc. Int. Conf. Biometrics, ICB, 2007.
    26. 26)
      • 20. Yang, S., Luo, P., Loy, C.C., et al: ‘From facial parts responses to face detection: a deep learning approach’. Proc. Int. Conf. Computer Vision, ICCV, 2015, pp. 36763684.
    27. 27)
      • 53. Loutas, E., Nikou, C., Pitas, I.: ‘An information theoretic approach to joint probabilistic face detection and tracking’. Proc. Int. Conf. Image Processing, ICIP, 2002, vol. 1, pp. I505–I–508.
    28. 28)
      • 51. Bing, X., Wei, Y., Chareonsak, C.: ‘Automatic focusing technique for face detection and face contour tracking’. IEEE Int. Workshop on Biomedical Circuits and Systems, December 2004, pp. S3/2-9S3/2-12.
    29. 29)
      • 34. Delmas, P., Eveno, N., Lievin, M.: ‘Towards robust lip tracking’. Proc. Int. Conf. Pattern Recognition, ICPR, 2002, vol. 2, pp. 528531.
    30. 30)
      • 23. Köstinger, M., Wohlhart, P., Roth, P.M., et al: ‘Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization’. Proc. Int. Conf. Computer Vision Workshops, ICCV, November 2011, pp. 21442151.
    31. 31)
      • 75. Valenti, R., Gevers, T.: ‘Robustifying eye center localization by head pose cues’. Proc. Int. Conf. Computer Vision and Pattern Recognition, CVPR, 2009.
    32. 32)
      • 30. Maurel, P., McGonigal, A., Keriven, R., et al: ‘3d model fitting for facial expression analysis under uncontrolled imaging conditions’. Proc. Int. Conf. Pattern Recognition, ICPR, December 2008, pp. 14.
    33. 33)
      • 1. Li, S.Z., Jain, A.K. (Eds.): ‘Handbook of face recognition’ (Springer Verlag, 2011, 2nd edn.).
    34. 34)
      • 78. Irani, M.: ‘Multi-frame correspondence estimation using subspace constraints’, Int. J. Comput. Vis., 2002, 48, (3), pp. 173194.
    35. 35)
      • 65. Shi, J., Tomasi, C.: ‘Good features to track’. Proc. Int. Conf. Computer Vision and Pattern Recognition, CVPR, 1994.
    36. 36)
      • 26. Liao, S., Jain, A.K., Li, S.Z.: ‘A fast and accurate unconstrained face detector’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 211223.
    37. 37)
      • 25. Li, H., Hua, G., Lin, Z., et al: ‘Probabilistic elastic part model for unsupervised face detector adaptation’. Proc. Int. Conf. Computer Vision, ICCV, December 2013, pp. 793800.
    38. 38)
      • 79. Taigman, Y., Yang, M., Ranzato, M., et al: ‘Deepface: closing the gap to human-level performance in face verification’. Proc. Conf. Computer Vision and Pattern Recognition, CVPR, June 2014, pp. 17011708.
    39. 39)
      • 12. Sung, K.-K., Poggio, T.: ‘Example-based learning for view-based human face detection’, IEEE Trans. Pattern Anal. Mach. Intell., 1998, 20, (1), pp. 3951.
    40. 40)
      • 50. Destrero, A., Odone, F., Verri, A.: ‘A system for face detection and tracking in unconstrained environments’. Proc. IEEE Conf. Advanced Video and Signal Based Surveillance, AVSS, September 2007, pp. 499504.
    41. 41)
      • 69. Karlsson, S.M., Bigun, J.: ‘Lip-motion events analysis and lip segmentation using optical flow’. Proc. IEEE Computer Vision and Pattern Recognition Biometrics Workshop, CVPRW, 2012.
    42. 42)
      • 39. Tao, J., Tan, Y.-P.: ‘A unified probabilistic approach to face detection and tracking’. IEEE Int. Symp. Circuits and Systems, SCAS, May 2005, vol. 4, pp. 37973800.
    43. 43)
      • 8. Parkhi, O.M., Vedaldi, A., Zisserman, A.: ‘Deep face recognition’. Proc. British Machine Vision Conf., BMVC, 2015.
    44. 44)
      • 9. Mathias, M., Benenson, R., Pedersoli, M., et al: ‘Face detection without bells and whistles’. Proc. European Conf. Computer Vision, ECCV, 2014, pp. 720735.
    45. 45)
      • 16. Lienhart, R., Kuranov, A., Pisarevsky, V.: ‘Empirical analysis of detection cascades of boosted classifiers for rapid object detection’. Proc. DAGM 25th Pattern Recognition Symp., 2003.
    46. 46)
      • 33. DeCarlo, D., Metaxas, D.: ‘Optical flow constraints on deformable models with applications to face tracking’, Int. J. Comput. Vis., 2000, 38, (2), pp. 99127.
    47. 47)
      • 44. Grest, D., Koch, R.: ‘Realtime multi-camera person tracking for immersive environments’. Proc. IEEE Workshop on Multimedia Signal Processing, September 2004, pp. 387390.
    48. 48)
      • 36. Chen, J., Tiddeman, B.: ‘Multi-cue facial feature detection and tracking’. Proc. Int. Conf. Image and Signal Processing, ICISP, 2008, pp. 356367.
    49. 49)
      • 3. Stallkamp, J., Ekenel, H.K., Stiefelhagen, R.: ‘Video-based face recognition on real-world data’. Proc. Int. Conf. Computer Vision, ICCV, October 2007, pp. 18.
    50. 50)
      • 10. Yang, M.-H., Kriegman, D., Ahuja, N.: ‘Detecting faces in images: a survey’, IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24, (1), pp. 3458.
    51. 51)
      • 56. Jagathishwaran, R., Ravichandran, K.S., Jayaraman, P.: ‘A survey on face detection and tracking’, World Appl. Sci. J., 2014, 29, pp. 140145.
    52. 52)
      • 32. Gast, E.R.: ‘A framework for real-time face and facial feature tracking using optical flow pre-estimation and template tracking’. MS thesis, Leiden University, 2010.
    53. 53)
      • 71. Yang, S., Luo, P., Loy, C.C., et al: ‘Wider face: a face detection benchmark’. Proc. Conf. Computer Vision and Pattern Recognition, CVPR, 2016.
    54. 54)
      • 70. Farneback, G.: ‘Two-frame motion estimation based on polynomial expansion’. Proc. Swedish Symp. Image Analysis, SSBA, 2003.
    55. 55)
      • 21. Zhang, C., Zhang, Z.: ‘Improving multiview face detection with multi-task deep convolutional neural networks’. Proc. IEEE Winter Conf. Applications of Computer Vision, March 2014, pp. 10361041.
    56. 56)
      • 80. Fischer, P., Dosovitskiy, A., Ilg, E., et al: ‘Flownet: learning optical flow with convolutional networks’. Int. Conf. Computer Vision, ICCV, 2015.
    57. 57)
      • 28. Dornaika, F., Ahlberg, J.: ‘Fast and reliable active appearance model search for 3-d face tracking’, IEEE Trans. Syst. Man Cybern. B, Cybern., 2004, 34, (4), pp. 18381853.
    58. 58)
      • 62. Galoogahi, H.K., Sim, T., Lucey, S.: ‘Correlation filters with limited boundaries’. Proc. Conf. Computer Vision and Pattern Recognition, CVPR, June 2015, pp. 46304638.
    59. 59)
      • 38. Comaniciu, D., Ramesh, V., Meer, P.: ‘Kernel-based object tracking’, IEEE Trans. Pattern Anal. Mach. Intell., 2003, 25, (5), pp. 564577.
    60. 60)
      • 43. Hiremath, P.S., Hiremath, M., Mahesh, R.: ‘Face detection and tracking in video sequence using fuzzy geometric face model and mean shift’, Int. J. Adv. Trends Comput. Sci. Eng., 2013, 2, (1), pp. 4146.
    61. 61)
      • 76. Jung, S.U., Nixon, M.S.: ‘Model-based feature refinement by ellipsoidal face tracking’. Proc. Int. Conf. Pattern Recognition, ICPR, November 2012, pp. 12091212.
    62. 62)
      • 35. Wu, Z., Aleksic, P.S., Katsaggelos, A.K.: ‘Lip tracking for mpeg-4 facial animation’. Proc. IEEE Int. Conf. Multimodal Interfaces, 2002, pp. 293298.
    63. 63)
      • 11. Rowley, H.A., Baluja, S., Kanade, T.: ‘Neural network-based face detection’, IEEE Trans. Pattern Anal. Mach. Intell., 1998, 20, (1), pp. 2338.
    64. 64)
      • 52. Mohabbati, B., Kasaei, S.: ‘Face localization and versatile tracking in wavelet domain’. Proc. IEEE Conf. Information and Communication Technologies, ICTTA, 2006, vol. 1, pp. 15521556.
    65. 65)
      • 45. Su, C., Huang, L.: ‘Spatio-temporal graphical-model-based multiple facial feature tracking’, EURASIP J. Appl. Signal Process., 2005, 2005, pp. 20912100.
    66. 66)
      • 7. Zhang, K., Zhang, Z., Li, Z., et al: ‘Joint face detection and alignment using multitask cascaded convolutional networks’, IEEE Signal Process. Lett., 2016, 23, (10), pp. 14991503.
    67. 67)
      • 57. Kristan, M., Pflugfelder, R., Leonardis, A., et al: ‘The visual object tracking VOT2014 challenge results’ (Springer International Publishing, Cham, 2015), pp. 191217.
    68. 68)
      • 15. Teferi, D., Bigun, J.: ‘Evaluation protocol for the dxm2vts database and performance comparison of face detection and face tracking on video’. Int. Conf. Pattern Recognition, ICPR, December 2008, pp. 14.
    69. 69)
      • 31. Roy-Chowdhury, A.-K., Xu, Y.: ‘Face tracking’, in Li, S.Z., Jain, A. (Eds.): ‘Encyclopedia of biometrics’ (Springer, 2015, 2nd edn.).
    70. 70)
      • 18. Li, H., Lin, Z., Shen, X., et al: ‘A convolutional neural network cascade for face detection’. Proc. Int. Conf. Computer Vision and Pattern Recognition, CVPR, June 2015, pp. 53255334.
    71. 71)
      • 66. Bigun, J., Granlund, G.H.: ‘Optimal orientation detection of linear symmetry’. Proc. Int. Conf. Computer Vision, ICCV, June 1987, pp. 433438.
    72. 72)
      • 5. Li, S.Z.: ‘Face detection’, in Li, S.Z., Jain, A. (Eds.): ‘Handbook of face recognition’ (Springer Verlag, 2011, 2nd edn.).
    73. 73)
      • 14. Viola, P., Jones, M.: ‘Robust real-time face detection’, Int. J. Comput. Vis., 2004, 57, (2), pp. 137154.
    74. 74)
      • 40. Huang, H.-P., Lin, C.-T.: ‘Multi-camshift for multi-view faces tracking and recognition’. Proc. IEEE Int. Conf. Robotics and Biomimetics, ROBIO, December 2006, pp. 13341339.
    75. 75)
      • 6. Ranftl, A., Alonso-Fernandez, F., Karlsson, S.: ‘Face tracking using optical flow’. Proc. Int. Conf. the Biometrics Special Interest Group, BIOSIG (Best Paper Award), September 2015, pp. 15.
    76. 76)
      • 61. Gaxiola, L.N., Díaz-Ramírez, V.H., Tapia, J.J., et al: ‘Robust face tracking with locally-adaptive correlation filtering’ (Springer International Publishing, Cham, 2014), pp. 925932.
    77. 77)
      • 22. Jain, V., Learned-Miller, E.G.: ‘FDDB: a benchmark for face detection in unconstrained settings’. Technical Report UMCS-2010-009, University of Massachusetts, Amherst, 2010.
    78. 78)
      • 19. Liu, Z., Luo, P., Wang, X., et al: ‘Deep learning face attributes in the wild’. Proc. Int. Conf. Computer Vision, ICCV, 2015, pp. 37303738.
    79. 79)
      • 24. Li, J., Wang, T., Zhang, Y.: ‘Face detection using surf cascade’. Proc. Int. Conf. Computer Vision Workshops, ICCV, November 2011, pp. 21832190.
    80. 80)
      • 58. Wu, Y., Lim, J., Yang, M.H.: ‘Object tracking benchmark’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 37, (9), pp. 18341848.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-bmt.2016.0202
Loading

Related content

content/journals/10.1049/iet-bmt.2016.0202
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address