In this study, we propose a regression forests-based cascaded method for face alignment. We build on the cascaded pose regression (CPR) framework and propose to use the regression forest as a primitive regressor. The regression forests are easier to train and naturally handle the over-fitting problem via averaging the outputs of the trees at each stage. We address the fact that the CPR approaches are sensitive to the shape initialisation; in contrast to using a number of blind initialisations and selecting the median values, we propose an intelligent shape initialisation scheme. More specifically, a large number of initialisations are propagated to a few early stages in the cascade, then only a proportion of them are propagated to the remaining cascades according to their convergence measurement. We evaluate the performance of the proposed approach on the challenging face alignment in the wild database and obtain superior or comparable performance with the state-of-the-art, in spite of the fact that we have utilised only the freely available public training images. More importantly, we show that the intelligent initialisation scheme makes the CPR framework more robust to unreliable initialisations that are typically produced by different face detections.

References

1. 1)
  - 1. Li, S.Z., Jain, A.K.: ‘Handbook of face recognition’ (Springer, 2011, 1st edn.).
2. 2)
  - 34. Burgos-Artizzu, X.P., Perona, P., Dollár, P.: ‘Robust face landmark estimation under occlusion’. Proc. IEEE Conf. Computer Vision, 2013.
3. 3)
  - 35. Jesorsky, O., Kirchberg, K., Frischholz, R.: ‘Robust face detection using the Hausdorff distance’. Proc. Audio-and Video-Based Biometric Person Authentication, 2001.
4. 4)
  - 38. Kostinger, M., Wohlhart, P., Roth, P.M., Bischof, H.: ‘Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization’. Proc. IEEE Int. Conf. Computer Vision Workshops, 2011, pp. 2144–2151.
5. 5)
  - X. Jiang , B. Mandal , A. Kot . Eigenfeature regularization and extraction in face recognition. IEEE Trans. Patt. Anal. Mach. Intell. , 3 , 383 - 394
6. 6)
  - 24. Amberg, B., Vetter, T.: ‘Optimal landmark detection using shape models and branch and bound’. Proc. IEEE Int. Conf. Computer Vision, 2011.
7. 7)
  - 9. Xiong, X., De la Torre, F.: ‘Supervised descent method and its applications to face alignment’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013.
8. 8)
  - 40. Le, V., Brandt, J., Lin, Z., Bourdev, L., Huang, T.S.: ‘Interactive facial feature localization’. Proc. European Conf. Computer Vision, 2012.
9. 9)
  - 38. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: ‘Multi-PIE’, Image Vis. Comput., 2010, 28, (5), pp. 807–813 (doi: 10.1016/j.imavis.2009.08.002).
10. 10)
  - 44. Viola, P., Jones, M.: ‘Rapid object detection using a boosted cascade of simple features’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
11. 11)
  - 39. Cootes, T.F., Ionita, M.C., Sauer, P.: ‘Robust and accurate shape model fitting using random forest regression voting’. Proc. European Conf. Computer Vision, 2012.
12. 12)
  - 18. Valstar, M., Martinez, B., Binefa, X., Pantic, M.: ‘Facial point detection using boosted regression and graph models’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
13. 13)
  - 14. Liao, C.T., Wu, Y.K., Lai, S.H.: ‘Locating facial feature points using support vector machines’. Int. Workshop on Cellular Neural Networks and their Applications, 2005.
14. 14)
  - 31. Saragih, J., Goecke, R.: ‘A nonlinear discriminative approach to AAM fitting’. Proc. IEEE Conf. Computer Vision, 2007.
15. 15)
  - 21. Cristinacce, D., Cootes, T.: ‘Boosted regression active shape models’. Proc. British Machine Vision Conf., 2007, pp. 880–889.
16. 16)
  - 3. Suhr, J.K., Eum, S., Jung, H.G., Li, G., Kim, G., Kim, J.: ‘Recognizability assessment of facial images for automated teller machine applications’, Pattern Recognit., 2012, 45, (5), pp. 1899–1914 (doi: 10.1016/j.patcog.2011.11.014).
17. 17)
  - 17. Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: ‘Localizing parts of faces using a consensus of exemplars’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
18. 18)
  - 12. Efraty, B., Huang, C., Shah, S.K., Kakadiaris, I.A.: ‘Facial landmark detection in uncontrolled conditions’. Proc. Int. Joint Conf. on Biometrics, 2011.
19. 19)
  - 13. Vukadinovic, D., Pantic, M.: ‘Fully automatic facial feature point detection using Gabor feature based boosted classifiers’. Proc. IEEE Int. Conf. Systems, Man and Cybernetics, 2005.
20. 20)
  - 33. Tzimiropoulos, G., Pantic, M.: ‘Optimization problems for fast AAM fitting in-the-wild’. Proc. IEEE Int. Conf. Computer Vision, 2013.
21. 21)
  - 19. Zhu, X., Ramanan, D.: ‘Face detection, pose estimation and landmark localization in the wild’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
22. 22)
  - 7. Cristinacce, D., Cootes, T.: ‘Feature detection and tracking with constrained local models’. Proc. British Machine Vision Conf., 2006.
23. 23)
  - 41. Sun, Y., Wang, X., Tang, X.: ‘Deep convolutional network cascade for facial point detection’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013.
24. 24)
  - 22. Martinez, B., Valstar, M., Binefa, X., Pantic, M.: ‘Local evidence aggregation for regression based facial point detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 35, (5), pp. 1149–1163 (doi: 10.1109/TPAMI.2012.205).
25. 25)
  - 36. Messer, K., Matas, J., Kittler, J., Luettin, J., Maitre, G.: ‘Xm2vtsdb: the extended m2vts database’. Proc. Second Int. Conf. on Audio and Video-based Biometric Person Authentication, 1999, pp. 965–966.
26. 26)
  - 28. Wu, Y., Wang, Z., Ji, Q.: ‘Facial feature tracking under varying facial expressions and face poses based on restricted Boltzmann machines’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013.
27. 27)
  - H. Zhaofeng , T. Tan , Z. Sun , X. Qiu . Toward accurate and fast iris segmentation for iris biometrics. IEEE Trans. Pattern Anal. Mach. Intell. , 1670 - 1684
28. 28)
  - 45. Asthana, A., Zafeiriou, S., Cheng, S., Pantic, M.: ‘Robust discriminative response map fitting with constrained local models’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013.
29. 29)
  - 29. Zhou, F., Brandt, J., Lin, Z.: ‘Exemplar-based graph matching for robust facial landmark localization’. Proc. IEEE Int. Conf. Computer Vision, 2013.
30. 30)
  - 32. Tresadern, P.A., Sauer, P., Cootes, T.F.: ‘Additive update predictors in active appearance models’. Proc. British Machine Vision Conf., 2010.
31. 31)
  - 15. Rapp, V., Senechal, T., Bailly, K., Prevost, L.: ‘Multiple kernel learning SVM and statistical validation for facial landmark detection’. Proc. IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2011.
32. 32)
  - 5. Li, H., Yu, J., Ye, Y., Bregler, C.: ‘Realtime facial animation with on-the-fly correctives’, ACM Trans. Graph., 2013, 32, (4), pp. 35–42.
33. 33)
  - 26. Yang, H., Patras, I.: ‘Face parts localization using structured-output regression forests’. Proc. Asian Conf. Computer Vision, 2012.
34. 34)
  - 27. Tan, X., Song, F., Zhou, Z.H., Chen, S.: ‘Enhanced pictorial structures for precise eye localization under in controlled conditions’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
35. 35)
  - 8. Cao, X., Wei, Y., Wen, F., Sun, J.: ‘Face alignment by explicit shape regression’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
36. 36)
  - 11. Dollár, P., Welinder, P., Perona, P.: ‘Cascaded pose regression’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
37. 37)
  - 30. Yang, H., Patras, I.: ‘Sieving regression forests votes for facial feature detection in the wild’. Proc. Int. Conf. Computer Vision, 2013.
38. 38)
  - 23. Dantone, M., Gall, J., Fanelli, G., Van Gool, L.: ‘Real-time facial feature detection using conditional regression forests’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
39. 39)
  - T.F. Cootes , G.J. Edwards , C.J. Taylor . Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. , 6 , 681 - 685
40. 40)
  - 25. Saragih, J.M., Lucey, S., Cohn, J.F.: ‘Face alignment through subspace constrained mean-shifts’. Proc. IEEE Int. Conf. Computer Vision, 2009.
41. 41)
  - 43. Yang, H., Patras, I.: ‘Privileged information-based conditional regression forests for facial feature detection’. Proc. IEEE Int. Conf. on Automatic Face and Gesture Recognition, 2013.
42. 42)
  - 20. Boddeti, V.N., Kanade, T., Kumar, B.V.: ‘Correlation filters for object alignment’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013.
43. 43)
  - 37. Kasinski, A., Florek, A., Schmidt, A.: ‘The PUT face database’, Image Process. Commun., 2008, 13, (3–4), pp. 59–64.
44. 44)
  - 23. Lee, C.-Y., Leou, J.-J., Hsiao, H.-H.: ‘Saliency-directed colour image segmentation using modified particle swarm optimization’, Signal Process., 2012, 92, (1), pp. 1–18 (doi: 10.1016/j.sigpro.2011.04.026).
45. 45)
  - 42. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: ‘A semi-automatic methodology for facial landmark annotation’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), 2013.
46. 46)
  - 16. Du, C., Wu, Q., Yang, J., Wu, Z.: ‘SVM based ASM for facial landmarks location’. Proc. IEEE Int. Conf. Computer and Information Technology, 2008.

Cascade of forests for face alignment

References

Related content