Survey on GAN-based face hallucination with its model development
Survey on GAN-based face hallucination with its model development
- Author(s): Heng Liu 1, 2 ; Xiaoyu Zheng 1 ; Jungong Han 3 ; Yuezhong Chu 1 ; Tao Tao 1
- DOI: 10.1049/iet-ipr.2018.6545
For access to this article, please select a purchase option:
Buy article PDF
Buy Knowledge Pack
IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.
Thank you
Your recommendation has been sent to your librarian.
- Author(s): Heng Liu 1, 2 ; Xiaoyu Zheng 1 ; Jungong Han 3 ; Yuezhong Chu 1 ; Tao Tao 1
-
-
View affiliations
-
Affiliations:
1:
Anhui University of Technology , Maxiang Road, Ma'anshan 243000 , People's Republic of China ;
2: Key Laboratory of Intelligent Perception and Systems for High Dimensional Information of Ministry of Education , Nanjing 210094 , People's Republic of China ;
3: School of Computing & Communications, Lancaster University , LA1 4YW , UK
-
Affiliations:
1:
Anhui University of Technology , Maxiang Road, Ma'anshan 243000 , People's Republic of China ;
- Source:
Volume 13, Issue 14,
12
December
2019,
p.
2662 – 2672
DOI: 10.1049/iet-ipr.2018.6545 , Print ISSN 1751-9659, Online ISSN 1751-9667
Face hallucination aims to produce a high-resolution face image from an input low-resolution face image, which is of great importance for many practical face applications, such as face recognition and face verification. Since the structure of the face image is complex and sensitive, obtaining a super-resolved face image is more difficult than generic image super-resolution. Recently, with great success in the high-level face recognition task, deep learning methods, especially generative adversarial networks (GANs), have also been applied to the low-level vision task – face hallucination. This work is to provide a model evolvement survey on GAN-based face hallucination. The principles of image resolution degradation and GAN-based learning are presented firstly. Then, a comprehensive review of the state-of-art GAN-based face hallucination methods is provided. Finally, the comparisons of these GAN-based face hallucination methods and the discussions of the related issues for future research direction are also provided.
Inspec keywords: face recognition; learning (artificial intelligence); image resolution
Other keywords: GAN-based learning; state-of-art GAN-based face hallucination; practical face applications; face verification; image resolution degradation; input low-resolution face image; high-level face recognition task; generic image super-resolution; super-resolved face image; high-resolution face image
Subjects: Knowledge engineering techniques; Image recognition; Other topics in statistics; Optical, image and video signal processing; Other topics in statistics; Computer vision and image processing techniques
References
-
-
1)
-
1. Wang, Q., Chen, M., Nie, F., et al: ‘Detecting coherent groups in crowd scenes by multiview clustering’, IEEE Trans. Pattern Anal. Mach. Intell., 2018, to be published., DOI: 10.1109/TPAMI.2018.2875002.
-
-
2)
-
2. Zheng, F., Shao, L.: ‘A winner-take-All strategy for improved object tracking’, IEEE Trans. Image Process., 2018, 27, (9), pp. 4302–4313.
-
-
3)
-
3. Wang, Q., Wan, J., Yuan, Y.: ‘Deep metric learning for crowdedness regression’, IEEE Trans. Circuit Syst. Video Technol., 2018, 28, (10), pp. 2633–2643.
-
-
4)
-
4. Wang, Q., Qin, Z., Nie, F., et al: ‘Spectral embedded adaptive neighbors clustering’, IEEE Trans. Neural Netw. Learn. Syst., 2018, (99), pp. 1–7, to be published., DOI: 10.1109/TPAMI.2018.2875002.
-
-
5)
-
5. Wang, Q., Wan, J., Nie, F., et al: ‘Hierarchical feature selection for random projection’, IEEE Trans. Neural Netw. Learn. Syst., 2018, to be published., DOI: 10.1109/TNNLS.2018.2868836.
-
-
6)
-
6. Zheng, F., Tang, Y., Shao, L.: ‘Hetero-manifold regularization for cross-modal hashing’, IEEE Trans. Pattern Anal. Mach. Intell., 2018, 40, (5), pp. 1059–1071.
-
-
7)
-
7. Baker, S., Kanade, T.: ‘Hallucinating faces’. Proc. Fourth IEEE Int. Conf. on Automatic Face and Gesture Recognition, Grenoble, France, 2000, pp. 83–88.
-
-
8)
-
8. Yang, J., Wright, J., Huang, T.S., et al: ‘Image super-resolution via sparse representation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 2861–2873.
-
-
9)
-
9. Liu, C., Shum, H.-Y., Freeman, W. T.: ‘Face hallucination: theory and practice’, Int. J. Comput. Vis., 2007, 75, (1), pp. 115–134.
-
-
10)
-
10. Yang, C.-Y., Liu, S., Yang, M.-H.: ‘Structured face hallucination’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 1099–1106.
-
-
11)
-
11. Zhu, S., Liu, S., Loy, C. C., et al: ‘Deep cascaded bi-network for face hallucination’. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 614–630.
-
-
12)
-
12. Dahl, R., Norouzi, M., Shlens, J.: ‘Pixel recursive super resolution’, arXiv preprint arXiv:.00783, 2017.
-
-
13)
-
13. van den Oord, A., Kalchbrenner, N., Espeholt, L., et al: ‘Conditional image generation with Pixelcnn decoders’. Proc. Conf. and Workshop on Neural Information Processing Systems, Barcelona, Spain, Dec 2016, pp. 4797–4805.
-
-
14)
-
14. Ledig, C., Theis, L., Huszár, F., et al: ‘Photo-realistic single image super-resolution using a generative adversarial network’. Proc. Int. Conf. on Computer Vision and Pattern Recogintion, Hawaii, USA, Jul 2017, pp. 105–114.
-
-
15)
-
15. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al: ‘Generative adversarial nets’. Proc. Conf. and Workshop on Neural Information Processing Systems, Montréal, Canada, Jul 2017, pp. 2672–2680.
-
-
16)
-
16. Arjovsky, M., Chintala, S., Bottou, L.: ‘Wasserstein Gan’, arXiv preprint arXiv:.07875, 2017.
-
-
17)
-
17. Qi, G.-J.: ‘Loss-sensitive generative adversarial networks on Lipschitz densities’, arXiv preprint arXiv:.06264, 2017.
-
-
18)
-
18. Bruna, J., Sprechmann, P., Lecun, Y.: ‘Super-resolution with deep convolutional sufficient statistics’, Comput. Sci., 2015, arXiv preprint arXiv:.05666, 2015.
-
-
19)
-
19. Dong, C., Loy, C. C., He, K., et al: ‘Image super-resolution using deep convolutional networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 295–307.
-
-
20)
-
20. Kim, J., Lee, J. K., Lee, K. M.: ‘Accurate image super-resolution using very deep convolutional networks’. IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 1646–1654.
-
-
21)
-
21. Yu, X., Porikli, F.: ‘Ultra-resolving face images by discriminative generative networks’. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 318–333.
-
-
22)
-
22. Chen, Z., Tong, Y.: ‘Face super-resolution through Wasserstein Gans’, arXiv preprint arXiv:.02438, 2017.
-
-
23)
-
23. Gulrajani, I., Ahmed, F., Arjovsky, M., et al: ‘Improved training of Wasserstein Gans’. Proc. Conf. and Workshop on Neural Information Processing Systems, Long Beach, USA, Jan 2017, pp. 5767–5777.
-
-
24)
-
24. Radford, A., Metz, L., Chintala, S.: ‘Unsupervised representation learning with deep convolutional generative adversarial networks’, arXiv preprint arXiv:.06434, 2015.
-
-
25)
-
25. Liu, Z., Luo, P., Wang, X., et al: ‘Deep learning face attributes in the wild’. IEEE Int. Conf. on Computer Vision, Santiago, Chile, 2015, pp. 3730–3738.
-
-
26)
-
26. Berthelot, D., Schumm, T., Metz, L.: ‘Began: boundary equilibrium generative adversarial networks’, arXiv preprint arXiv:.10717, 2017.
-
-
27)
-
27. Huang, B., Chen, W., Wu, X., et al: ‘High-quality face image Sr using conditional generative adversarial networks’, arXiv preprint arXiv:.00737, 2017.
-
-
28)
-
28. Ronneberger, O., Fischer, P., Brox, T.: ‘U-Net: convolutional networks for biomedical image segmentation’. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
-
-
29)
-
29. Yu, X., Porikli, F.: ‘Hallucinating very low-resolution unaligned and noisy face images by transformative discriminative autoencoders’. Proc. Int. Conf. on Computer Vision and Pattern Recogintion, Hawaii, USA, Jul 2017, pp. 5367–5375.
-
-
30)
-
30. Yu, X., Fernando, B., Hartley, R., et al: ‘Super-resolving very low-resolution face images with supplementary attributes’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake, USA, 2018, pp. 908–917.
-
-
31)
-
31. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:.1556, 2014.
-
-
32)
-
32. Bulat, A., Tzimiropoulos, G.: ‘Super-Fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with Gans’, arXiv preprint arXiv:.02765, 2017.
-
-
33)
-
33. Bulat, A., Tzimiropoulos, G.: ‘How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks)’. Int. Conf. on Computer Vision, Venice, Italy, 2017, p. 4.
-
-
34)
-
34. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 770–778.
-
-
35)
-
35. Chen, Y., Tai, Y., Liu, X., et al: ‘Fsrnet: end-to-end learning face super-resolution with facial priors’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake, USA, 2018, pp. 2492–2501.
-
-
36)
-
36. Newell, A., Yang, K., Deng, J.: ‘Stacked hourglass networks for human pose estimation’. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 483–499.
-
-
37)
-
37. Chen, Y., Shen, C., Wei, X. S., et al: ‘Adversarial Posenet: a structure-aware convolutional network for human pose estimation’. Proc. IEEE Int. Conf. on Computer Vision, Venice, Italy, Oct 2017, pp. 1221–1230.
-
-
38)
-
38. Yu, X., Fernando, B., Ghanem, B., et al: ‘Face super-resolution guided by facial component heatmaps’. Proc. of the European Conf. on Computer Vision (ECCV), Munich, Germany, 2018, pp. 217–233.
-
-
39)
-
39. Collobert, R., Kavukcuoglu, K., Farabet, C.: ‘Torch7: an matlab-like environment for machine learning’. BigLearn, NIPS Workshop, Granada, Spain, 2011.
-
-
40)
-
40. Girija, S.S.: ‘Tensorflow: large-scale machine learning on heterogeneous distributed systems’, Software available from tensorflow.org, 2016.
-
-
41)
-
41. Zhu, X., Lei, Z., Liu, X., et al: ‘Face alignment across large poses: a 3d solution’. Proc. of the IEEE Conf. on computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 146–155.
-
-
42)
-
42. Koestinger, M., Wohlhart, P., Roth, P. M., et al: ‘Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization’. 2011 IEEE Int. Conf. on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 2011, pp. 2144–2151.
-
-
43)
-
43. ‘Pytorch’. Available at https://github.com/pytorch/pytorch, accessed October 2018.
-
-
44)
-
44. Le, V., Brandt, J., Lin, Z., et al: ‘Interactive facial feature localization’. European Conf. on Computer Vision, Florence, Italy, 2012, pp. 679–692.
-
-
45)
-
45. Wang, Z., Bovik, A.C., Sheikh, H.R., et al: ‘Image quality assessment: from error visibility to structural similarity’, IEEE Trans. Image Process., 2004, 13, (4), pp. 600–612.
-
-
46)
-
46. Wang, Y., Xie, Z., Xu, K., et al: ‘An efficient and effective convolutional auto-encoder extreme learning machine network for 3d feature learning’, Neurocomputing, 2016, 174, pp. 988–998.
-
-
47)
-
47. Tissera, M. D., McDonnell, M. D.: ‘Deep extreme learning machines: supervised autoencoding architecture for classification’, Neurocomputing, 2016, 174, pp. 42–49.
-
-
1)

Related content
