In this study, an automatic face image annotation method is proposed by aligning faces with different expressions to an annotated neutral face. This work is useful in reducing tedious manual work for labelling image data in large databases. However, it is challenging because of the appearance variations caused by non-rigid face deformations under various expressions. Unlike some conventional approaches acquiring sufficient image templates to model the query appearance, only a single given template is necessary for the proposed method. The authors address the problem through dense image alignment. Specifically, image warping in the alignment process is constrained by prior knowledge about facial shape deformation. The proposed method is independent of the appearance model, and is available for unseen faces. In addition, to initialise warping parameters, the authors present a robust patch-based estimation method. Context information for feature points is carefully modelled to propagate the searching path for local patch matching. The face annotation experiments are performed on some large expressions, with noisy image qualities and in low image resolutions. Comparison results with conventional methods demonstrate the proposed method's superiority on both accuracy and robustness.

References

1. 1)
  - 5. Saragih, J., Goecke, R.: ‘Learning active appearance models from image sequences’, VisHCI, 2006, 56, pp. 51–60.
2. 2)
  - 2. Zhou, M., Liang, L., Sun, J., Wang, Y.: ‘AAM based face tracking with temporal matching and face segmentation’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2010, pp. 701–708.
3. 3)
  - 8. Cao, X., Wei, Y., Wen, F., Sun, J.: ‘Face alignment by explicit shape regression’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2012, pp. 2887–2894.
4. 4)
  - 11. Yang, Y., Yuan, Z., Zheng, N., et al.: ‘Interactive facial-geometric-feature animation for generating expressions of novel faces’, IEICE Trans. Inf. Syst., 2011, E94-D, (5), pp. 1099–1108.
5. 5)
  - 16. Liu, X.: ‘Discriminative face alignment’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (11), pp. 1941–1954.
6. 6)
  - 17. Liao, W., Fidaleo, D., Medioni, G.: ‘Integrating multiple visual cues for robust real-time 3D face tracking’. Proc. IEEE Workshop on Analysis and Model of Faces on Gestures, 2007, pp. 109–123.
7. 7)
  - 15. Liang, L., Xiao, R., Wen, F., Sun, J.: ‘Face alignment via component-based discriminative search’. European Conf. on Computer Vision, 2008, pp. 72–85.
8. 8)
  - 21. Kanade, T., Cohn, J.F., Tian, Y.: ‘Comprehensive database for facial expression analysis’, Autom. Face Gesture Recognit., 2000, pp. 46–53. http://www.pitt.edu/j~effcohn/CKandCK+.htm.
9. 9)
  - 13. Kazemi, V., Sullivan, J.: ‘Face alignment with part-based modeling’. Proc. British Machine Vision Conf., 2011, pp. 27.1–27.10.
10. 10)
  - 20. Lowe, D.: ‘Object recognition from local scale-invariant features’. Int. Conf. Computer Vision, 1999, pp. 1150–1157.
11. 11)
  - 19. Cootes, T., Taylor, C., Graham, J.: ‘Active shape models-their training and application’, Comput. Vis. Image Underst., 1995, 61, (1), pp. 38–59.
12. 12)
  - 12. Baker, S., Matthews, I.: ‘Lucas-kanada 20 years on: a unifying framework’, Int. J. Comput. Vis., 2004, 56, (3), pp. 221–255.
13. 13)
  - 9. Liang, L., Wen, F., Xu, Y., et al.: ‘Accurate face alignment using shape constrained Markov network’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2006, pp. 1313–1319.
14. 14)
  - 4. Walker, K., Cootes, T., Taylor, C.: ‘Automatically building appearance models from image sequrences using salient features’, Image Vis. Comput., 2002, 20, (5–6), pp. 435–440.
15. 15)
  - 14. Cristinacce, D., Cootes, T.: ‘Feature detection and tracking with constrained local models’. Proc. British Machine Vision Conf., 2006, pp. 929–938.
16. 16)
  - 7. Gross, R., Matthews, I., Baker, S.: ‘Generic vs. person specific active appearance models’, Image Vis. Comput., 2005, 23, (11), pp. 1080–1093.
17. 17)
  - 6. Cootes, T., Edwards, G., Taylor, C.: ‘Active appearance models’, IEEE Trans. Pattern Anal. Mach. Intell., 2001, 23, (6), pp. 681–685.
18. 18)
  - 18. Wang, Y., Lucey, S., Cohn, J.: ‘Non-rigid object alignment with a mismatch template based on exhaustive local search’. IEEE Workshop on Non-rigid Registration and Tracking through Learning, 2007, pp. 1–8.
19. 19)
  - 3. Liu, X.: ‘Video-based face model fitting using adaptive active appearance model’, Image Vis. Comput., 2010, 28, (7), pp. 1162–1172.
20. 20)
  - 10. Du, Y., Lin, X.: ‘Emotional facial expression model building’, Pattern Recognit. Lett., 2003, 24, (16), pp. 2923–2934.
21. 21)
  - 1. Mosquera, L., Castro, J.L.: ‘Performance of active appearance model-based pose-robust face recognition’, IET Comput. Vis., 2011, 5, (6), pp. 348–357.

Automatic face image annotation based on a single template with constrained warping deformation

References

Related content