http://iet.metastore.ingenta.com
1887

Survey on GAN-based face hallucination with its model development

Survey on GAN-based face hallucination with its model development

For access to this article, please select a purchase option:

Buy article PDF
$19.95
(plus tax if applicable)
Add to cart
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)
Add to cart

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Image Processing — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

  • Author(s): Heng Liu 1, 2 ; Xiaoyu Zheng 1 ; Jungong Han 3 ; Yuezhong Chu 1 ; Tao Tao 1
    • View affiliations
  • Source: Volume 13, Issue 14, 12 December 2019, p. 2662 – 2672
    DOI:  10.1049/iet-ipr.2018.6545 , Print ISSN 1751-9659, Online ISSN 1751-9667
© The Institution of Engineering and Technology
Received 26/11/2018, Accepted 05/02/2019, Revised 21/01/2019, Published 15/02/2019

Face hallucination aims to produce a high-resolution face image from an input low-resolution face image, which is of great importance for many practical face applications, such as face recognition and face verification. Since the structure of the face image is complex and sensitive, obtaining a super-resolved face image is more difficult than generic image super-resolution. Recently, with great success in the high-level face recognition task, deep learning methods, especially generative adversarial networks (GANs), have also been applied to the low-level vision task – face hallucination. This work is to provide a model evolvement survey on GAN-based face hallucination. The principles of image resolution degradation and GAN-based learning are presented firstly. Then, a comprehensive review of the state-of-art GAN-based face hallucination methods is provided. Finally, the comparisons of these GAN-based face hallucination methods and the discussions of the related issues for future research direction are also provided.

Inspec keywords: face recognition; learning (artificial intelligence); image resolution

Other keywords: GAN-based learning; state-of-art GAN-based face hallucination; practical face applications; face verification; image resolution degradation; input low-resolution face image; high-level face recognition task; generic image super-resolution; super-resolved face image; high-resolution face image

Subjects: Knowledge engineering techniques; Image recognition; Other topics in statistics; Optical, image and video signal processing; Other topics in statistics; Computer vision and image processing techniques

References

    1. 1)
      • 1. Wang, Q., Chen, M., Nie, F., et al: ‘Detecting coherent groups in crowd scenes by multiview clustering’, IEEE Trans. Pattern Anal. Mach. Intell., 2018, to be published., DOI: 10.1109/TPAMI.2018.2875002.
    2. 2)
      • 2. Zheng, F., Shao, L.: ‘A winner-take-All strategy for improved object tracking’, IEEE Trans. Image Process., 2018, 27, (9), pp. 4302–4313.
    3. 3)
      • 3. Wang, Q., Wan, J., Yuan, Y.: ‘Deep metric learning for crowdedness regression’, IEEE Trans. Circuit Syst. Video Technol., 2018, 28, (10), pp. 2633–2643.
    4. 4)
      • 4. Wang, Q., Qin, Z., Nie, F., et al: ‘Spectral embedded adaptive neighbors clustering’, IEEE Trans. Neural Netw. Learn. Syst., 2018, (99), pp. 1–7, to be published., DOI: 10.1109/TPAMI.2018.2875002.
    5. 5)
      • 5. Wang, Q., Wan, J., Nie, F., et al: ‘Hierarchical feature selection for random projection’, IEEE Trans. Neural Netw. Learn. Syst., 2018, to be published., DOI: 10.1109/TNNLS.2018.2868836.
    6. 6)
      • 6. Zheng, F., Tang, Y., Shao, L.: ‘Hetero-manifold regularization for cross-modal hashing’, IEEE Trans. Pattern Anal. Mach. Intell., 2018, 40, (5), pp. 1059–1071.
    7. 7)
      • 7. Baker, S., Kanade, T.: ‘Hallucinating faces’. Proc. Fourth IEEE Int. Conf. on Automatic Face and Gesture Recognition, Grenoble, France, 2000, pp. 83–88.
    8. 8)
      • 8. Yang, J., Wright, J., Huang, T.S., et al: ‘Image super-resolution via sparse representation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 2861–2873.
    9. 9)
      • 9. Liu, C., Shum, H.-Y., Freeman, W. T.: ‘Face hallucination: theory and practice’, Int. J. Comput. Vis., 2007, 75, (1), pp. 115–134.
    10. 10)
      • 10. Yang, C.-Y., Liu, S., Yang, M.-H.: ‘Structured face hallucination’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 1099–1106.
    11. 11)
      • 11. Zhu, S., Liu, S., Loy, C. C., et al: ‘Deep cascaded bi-network for face hallucination’. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 614–630.
    12. 12)
      • 12. Dahl, R., Norouzi, M., Shlens, J.: ‘Pixel recursive super resolution’, arXiv preprint arXiv:.00783, 2017.
    13. 13)
      • 13. van den Oord, A., Kalchbrenner, N., Espeholt, L., et al: ‘Conditional image generation with Pixelcnn decoders’. Proc. Conf. and Workshop on Neural Information Processing Systems, Barcelona, Spain, Dec 2016, pp. 4797–4805.
    14. 14)
      • 14. Ledig, C., Theis, L., Huszár, F., et al: ‘Photo-realistic single image super-resolution using a generative adversarial network’. Proc. Int. Conf. on Computer Vision and Pattern Recogintion, Hawaii, USA, Jul 2017, pp. 105–114.
    15. 15)
      • 15. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al: ‘Generative adversarial nets’. Proc. Conf. and Workshop on Neural Information Processing Systems, Montréal, Canada, Jul 2017, pp. 2672–2680.
    16. 16)
      • 16. Arjovsky, M., Chintala, S., Bottou, L.: ‘Wasserstein Gan’, arXiv preprint arXiv:.07875, 2017.
    17. 17)
      • 17. Qi, G.-J.: ‘Loss-sensitive generative adversarial networks on Lipschitz densities’, arXiv preprint arXiv:.06264, 2017.
    18. 18)
      • 18. Bruna, J., Sprechmann, P., Lecun, Y.: ‘Super-resolution with deep convolutional sufficient statistics’, Comput. Sci., 2015, arXiv preprint arXiv:.05666, 2015.
    19. 19)
      • 19. Dong, C., Loy, C. C., He, K., et al: ‘Image super-resolution using deep convolutional networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 295–307.
    20. 20)
      • 20. Kim, J., Lee, J. K., Lee, K. M.: ‘Accurate image super-resolution using very deep convolutional networks’. IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 1646–1654.
    21. 21)
      • 21. Yu, X., Porikli, F.: ‘Ultra-resolving face images by discriminative generative networks’. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 318–333.
    22. 22)
      • 22. Chen, Z., Tong, Y.: ‘Face super-resolution through Wasserstein Gans’, arXiv preprint arXiv:.02438, 2017.
    23. 23)
      • 23. Gulrajani, I., Ahmed, F., Arjovsky, M., et al: ‘Improved training of Wasserstein Gans’. Proc. Conf. and Workshop on Neural Information Processing Systems, Long Beach, USA, Jan 2017, pp. 5767–5777.
    24. 24)
      • 24. Radford, A., Metz, L., Chintala, S.: ‘Unsupervised representation learning with deep convolutional generative adversarial networks’, arXiv preprint arXiv:.06434, 2015.
    25. 25)
      • 25. Liu, Z., Luo, P., Wang, X., et al: ‘Deep learning face attributes in the wild’. IEEE Int. Conf. on Computer Vision, Santiago, Chile, 2015, pp. 3730–3738.
    26. 26)
      • 26. Berthelot, D., Schumm, T., Metz, L.: ‘Began: boundary equilibrium generative adversarial networks’, arXiv preprint arXiv:.10717, 2017.
    27. 27)
      • 27. Huang, B., Chen, W., Wu, X., et al: ‘High-quality face image Sr using conditional generative adversarial networks’, arXiv preprint arXiv:.00737, 2017.
    28. 28)
      • 28. Ronneberger, O., Fischer, P., Brox, T.: ‘U-Net: convolutional networks for biomedical image segmentation’. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
    29. 29)
      • 29. Yu, X., Porikli, F.: ‘Hallucinating very low-resolution unaligned and noisy face images by transformative discriminative autoencoders’. Proc. Int. Conf. on Computer Vision and Pattern Recogintion, Hawaii, USA, Jul 2017, pp. 5367–5375.
    30. 30)
      • 30. Yu, X., Fernando, B., Hartley, R., et al: ‘Super-resolving very low-resolution face images with supplementary attributes’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake, USA, 2018, pp. 908–917.
    31. 31)
      • 31. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:.1556, 2014.
    32. 32)
      • 32. Bulat, A., Tzimiropoulos, G.: ‘Super-Fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with Gans’, arXiv preprint arXiv:.02765, 2017.
    33. 33)
      • 33. Bulat, A., Tzimiropoulos, G.: ‘How far are we from solving the 2d & 3d face alignment problem? (and a dataset of 230,000 3d facial landmarks)’. Int. Conf. on Computer Vision, Venice, Italy, 2017, p. 4.
    34. 34)
      • 34. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 770–778.
    35. 35)
      • 35. Chen, Y., Tai, Y., Liu, X., et al: ‘Fsrnet: end-to-end learning face super-resolution with facial priors’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake, USA, 2018, pp. 2492–2501.
    36. 36)
      • 36. Newell, A., Yang, K., Deng, J.: ‘Stacked hourglass networks for human pose estimation’. European Conf. on Computer Vision, Amsterdam, Netherlands, 2016, pp. 483–499.
    37. 37)
      • 37. Chen, Y., Shen, C., Wei, X. S., et al: ‘Adversarial Posenet: a structure-aware convolutional network for human pose estimation’. Proc. IEEE Int. Conf. on Computer Vision, Venice, Italy, Oct 2017, pp. 1221–1230.
    38. 38)
      • 38. Yu, X., Fernando, B., Ghanem, B., et al: ‘Face super-resolution guided by facial component heatmaps’. Proc. of the European Conf. on Computer Vision (ECCV), Munich, Germany, 2018, pp. 217–233.
    39. 39)
      • 39. Collobert, R., Kavukcuoglu, K., Farabet, C.: ‘Torch7: an matlab-like environment for machine learning’. BigLearn, NIPS Workshop, Granada, Spain, 2011.
    40. 40)
      • 40. Girija, S.S.: ‘Tensorflow: large-scale machine learning on heterogeneous distributed systems’, Software available from tensorflow.org, 2016.
    41. 41)
      • 41. Zhu, X., Lei, Z., Liu, X., et al: ‘Face alignment across large poses: a 3d solution’. Proc. of the IEEE Conf. on computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 146–155.
    42. 42)
      • 42. Koestinger, M., Wohlhart, P., Roth, P. M., et al: ‘Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization’. 2011 IEEE Int. Conf. on Computer Vision Workshops (ICCV Workshops), Barcelona, Spain, 2011, pp. 2144–2151.
    43. 43)
      • 43. ‘Pytorch’. Available at https://github.com/pytorch/pytorch, accessed October 2018.
    44. 44)
      • 44. Le, V., Brandt, J., Lin, Z., et al: ‘Interactive facial feature localization’. European Conf. on Computer Vision, Florence, Italy, 2012, pp. 679–692.
    45. 45)
      • 45. Wang, Z., Bovik, A.C., Sheikh, H.R., et al: ‘Image quality assessment: from error visibility to structural similarity’, IEEE Trans. Image Process., 2004, 13, (4), pp. 600–612.
    46. 46)
      • 46. Wang, Y., Xie, Z., Xu, K., et al: ‘An efficient and effective convolutional auto-encoder extreme learning machine network for 3d feature learning’, Neurocomputing, 2016, 174, pp. 988–998.
    47. 47)
      • 47. Tissera, M. D., McDonnell, M. D.: ‘Deep extreme learning machines: supervised autoencoding architecture for classification’, Neurocomputing, 2016, 174, pp. 42–49.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.6545
Loading

Related content

content/journals/10.1049/iet-ipr.2018.6545
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address