Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Blind text images deblurring based on a generative adversarial network

  • Author(s): Qing Qi 1, 2  and  Jichang Guo 1
    • View affiliations
  • Source: Volume 13, Issue 14, 12 December 2019, p. 2850 – 2858
    DOI:  10.1049/iet-ipr.2018.6697 , Print ISSN 1751-9659, Online ISSN 1751-9667
© The Institution of Engineering and Technology
Received 31/12/2018, Accepted 25/09/2019, Revised 15/07/2019, Published 02/10/2019

Recently, text images deblurring has achieved advanced development. Unlike previous methods based on hand-crafted priors or assume specific kernel, the authors recognise the text deblurring problem as a semantic generation task, which can be achieved by a generative adversarial network. The structure is an essential property of text images; thus, they propose a structural loss function and a detailed loss function to regularise the recovery of text images. Furthermore, they learn from the coarse-to-fine strategy and present a multi-scale generator, which is utilised for sharpening the generated text images. The model has a robust capability of generating realistic latent images with photo-quality effect. Extensive experiments on the synthetic and real-world blurry images have shown that the proposed network is comparable to the state-of-the-art methods.

Inspec keywords: learning (artificial intelligence); image restoration; text analysis

Other keywords: realistic latent images; detailed loss function; text deblurring problem; multiscale generator; generated text images; text images deblurring; blind text images; generative adversarial network; semantic generation task; specific kernel; hand-crafted priors; real-world blurry images; structural loss function

Subjects: Optical, image and video signal processing; Computer vision and image processing techniques; Document processing and analysis techniques; Knowledge engineering techniques

References

    1. 1)
      • 28. Ren, W., Cao, X., Pan, J., et al: ‘Image deblurring via enhanced low-rank prior’, IEEE Trans. Image Process., 2016, 25, (7), pp. 3426–3437.
    2. 2)
      • 24. Joshi, N., Szeliski, R., Kriegman, D.J.: ‘Psf estimation using sharp edge prediction’. Computer Vision and Pattern Recognition, Anchorage, USA, 2008, pp. 1–8.
    3. 3)
      • 13. Xu, J., Deng, C., Liu, X., et al: ‘Image super-resolution based on sparse representation with joint constraints’. Int. Conf. Internet Multimedia Computing and Service, Xiamen, China, 2014, pp. 10-12.
    4. 4)
      • 37. Su, S., Delbracio, M., Wang, J., et al: ‘Deep video deblurring for hand-held cameras’. Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, 2017, p. 6.
    5. 5)
      • 44. Kingma, D.P., Ba, J.: ‘Adam: a method for stochastic optimization’. arXiv, 2014, arXiv:1412.6980.
    6. 6)
      • 15. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al: ‘Generative adversarial nets’. Advances in Neural Information Processing Systems, Montreal, Canada, 2014, pp. 2672–2680.
    7. 7)
      • 40. Ronneberger, O., Fischer, P., Brox, T.: ‘U-net: convolutional networks for biomedical image segmentation’. Int. Conf. Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
    8. 8)
      • 11. Pan, J., Hu, Z., Su, Z., et al: ‘l_0-regularized intensity and gradient prior for deblurring text images and beyond’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (2), pp. 342–355.
    9. 9)
      • 9. Epshtein, B., Ofek, E., Wexler, Y.: ‘Detecting text in natural scenes with stroke width transform’. Computer Vision and Pattern Recognition, San Francisco, USA, 2010, pp. 2963–2970.
    10. 10)
      • 27. Hacohen, Y., Shechtman, E., Lischinski, D.: ‘Deblurring by example using dense correspondence’. Int. Conf. Computer Vision, Sydney, Australia, 2013, pp. 2384–2391.
    11. 11)
      • 38. Wieschollek, P., Hirsch, M., Schölkopf, B., et al: ‘Learning blind motion deblurring’. Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 231–240.
    12. 12)
      • 16. Johnson, J., Alahi, A., Fei-Fei, L.: ‘Perceptual losses for real-time style transfer and super-resolution’. European Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 694–711.
    13. 13)
      • 41. Ulyanov, D., Vedaldi, A., Lempitsky, V..: ‘Instance normalization: the missing ingredient for fast stylization’. arXiv preprint, 2016, arXiv:1607.08022.
    14. 14)
      • 3. Xu, L., Zheng, S., Jia, J.: ‘Unnatural L0 sparse representation for natural image deblurring’. Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 1107–1114.
    15. 15)
      • 34. Dong, C., Loy, C.C., He, K., et al: ‘Image super-resolution using deep convolutional networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 295–307.
    16. 16)
      • 31. Yan, R., Shao, L.: ‘Blind image blur estimation via deep learning’, IEEE Trans. Image Process., 2016, 25, (4), pp. 1910–1921.
    17. 17)
      • 6. Pan, J, Sun, D, Pfister, H, et al: ‘Blind image deblurring using dark channel prior’. Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 1628–1636.
    18. 18)
      • 10. Cao, X., Ren, W., Zuo, W., et al: ‘Scene text deblurring using text-specific multiscale dictionaries’, IEEE Trans. Image Process., 2015, 24, (4), pp. 1302–1314.
    19. 19)
      • 4. Nah, S., Kim, T.H., Lee, K.M.: ‘Deep multi-scale convolutional neural network for dynamic scene deblurring’. Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, 2017, pp. 257–265.
    20. 20)
      • 29. Schuler, C.J., Christopher-Burger, H., Harmeling, S., et al: ‘A machine learning approach for non-blind image deconvolution’. Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 1067–1074.
    21. 21)
      • 1. Cho, S., Lee, S.: ‘Fast motion deblurring’, ACM Trans. Graph. (TOG), 2009, 28, (5), p. 145.
    22. 22)
      • 2. Krishnan, D., Tay, T., Fergus, R.: ‘Blind deconvolution using a normalized sparsity measure’. Computer Vision and Pattern Recognition, Colorado, USA, 2011, pp. 233–240.
    23. 23)
      • 21. Levin, A., Weiss, Y., Durand, F., et al: ‘Understanding and evaluating blind deconvolution algorithms’. Computer Vision and Pattern Recognition, Miami, USA, 2009, pp. 1964–1971.
    24. 24)
      • 25. Sun, L., Cho, S., Wang, J., et al: ‘Edge-based blur kernel estimation using patch priors’. Int. Conf. Computational Photography, Massachusetts, USA, 2013, pp. 1–8.
    25. 25)
      • 20. Fergus, R., Singh, B., Hertzmann, A., et al: ‘Removing camera shake from a single photograph’. ACM transactions on graphics, 2006, 25, (3), pp. 787–794.
    26. 26)
      • 8. Cho, H., Wang, J., Lee, S.: ‘Text image deblurring using text-specific properties’. European Conf. Computer Vision, Berlin, Germany, 2012, pp. 524–537.
    27. 27)
      • 12. Deng, C., Xu, J., Zhang, K., et al: ‘Similarity constraints-based structured output regression machine: an approach to image super-resolution’, IEEE Trans. Neural Netw. Learn. Syst., 2015, 27, (12), pp. 2472–2485.
    28. 28)
      • 30. Schuler, C.J., Hirsch, M., Harmeling, S., et al: ‘Learning to deblur’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (7), pp. 1439–1451.
    29. 29)
      • 39. Tao, X., Gao, H., Liao, R., et al: ‘Detail-revealing deep video super-resolution’. Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 22–29.
    30. 30)
      • 7. Chen, X., He, X., Yang, J., et al: ‘An effective document image deblurring algorithm’. Computer Vision and Pattern Recognition, Providence, USA, 2011, pp. 369–376.
    31. 31)
      • 14. Hradiš, M., Kotera, J., Zemck, P., et al: ‘Convolutional neural networks for direct text deblurring’. British Machine Vision Conf., Swansea, UK, 2015, vol. 10, p. 2.
    32. 32)
      • 23. Xu, L., Jia, J.: ‘Two-phase kernel estimation for robust motion deblurring’. European Conf. Computer Vision, Crete, Greece, 2010, pp. 157–170.
    33. 33)
      • 5. Zhong, L., Cho, S., Metaxas, D., et al: ‘Handling noise in single image deblurring using directional filters’. Computer Vision and Pattern Recognition, Portland, USA, 2013, pp. 612–619.
    34. 34)
      • 19. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’. arXiv, 2014, arXiv:1409.1556.
    35. 35)
      • 17. Ledig, C., Theis, L., Huszĺćr, F., et al: ‘Photo-realistic single image super-resolution using a generative adversarial network’. Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, 2017, pp. 4681–4690.
    36. 36)
      • 36. Mao, X., Shen, C., Yang, Y.B.: ‘Image restoration using very deep convolutional encoder–decoder networks with symmetric skip connections’. Advances in Neural Information Processing Systems, Barcelona, Spain, 2016, pp. 2802–2810.
    37. 37)
      • 42. Xu, X., Sun, D., Pan, J., et al: ‘Learning to super-resolve blurry face and text images’. Int. Conf. Computer Vision, Venice, Italy, 2017, pp. 251–260.
    38. 38)
      • 18. Isola, P., Zhu, J.Y., Zhou, T., et al: ‘Image-to-image translation with conditional adversarial networks’. Computer Vision and Pattern Recognition, Honolulu, Hawaii, USA, 2017, pp. 5967–5976.
    39. 39)
      • 32. Ayan, C.: ‘A neural approach to blind motion deblurring’. European Conf. Computer Vision, Amsterdam, The Netherlands, 2016, pp. 221–235.
    40. 40)
      • 26. Pan, J., Hu, Z., Su, Z., et al: ‘Deblurring face images with exemplars’. European Conf. Computer Vision, Zurich, Switzerland, 2014, pp. 47–62.
    41. 41)
      • 33. Xu, L., Ren, J.S., Liu, C., et al: ‘Deep convolutional neural network for image deconvolution’. Advances in Neural Information Processing Systems, Barcelona, Spain, 2014, pp. 1790–1798.
    42. 42)
      • 35. Reshma Vijay, V.J., Deepa, P.L.: ‘Image deblurring using convolutional neural network’, IOSR J. Electron. Commun. Eng., 2016, 11, pp. 7–12.
    43. 43)
      • 22. Zhang, H., Yang, J., Zhang, Y., et al: ‘Sparse representation based blind image deblurring’. Int. Conf. Multimedia and Expo, Barcelona, Spain, 2011, pp. 1–6.
    44. 44)
      • 43. Maas, A.L., Hannun, A.Y., Ng, A.Y.: ‘Rectifier nonlinearities improve neural network acoustic models’. Int. Conf. Mach. Learn., Atlanta, USA, 2013, vol. 1, no. 30, p. 3.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.6697
Loading

Related content

content/journals/10.1049/iet-ipr.2018.6697
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address