Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Speedy and accurate image super-resolution via deeply recursive CNN with skip connection and network in network

© The Institution of Engineering and Technology
Received 23/07/2018, Accepted 18/03/2019, Revised 03/02/2019, Published 22/03/2019

The single image super-resolution (SISR) methods based on the deep convolutional neural network (CNN) have recently achieved significant improvements in accuracy, advancing the state of the art. However, these deeper models are computationally expensive and require a large number of parameters. Accordingly, they demand more memory and are unsuitable for on-chip devices. In this study, a novel SISR method using a deeply recursive CNN with skip connections and a network in network structure is proposed. The deeply recursive CNN with skip connections is adopted for the image feature extraction at both local and global levels. Parallelised 1 × 1 CNNs, usually called a network in network structure, are adopted for image reconstruction. Specifically, recursive learning is utilised to control the number of model parameters needed and residual learning is used to ease the difficulty of training. The proposed method performs favourably against the state-of-the-art methods in terms of computational speed and accuracy. It significantly outperforms the previous methods by a large margin, while demanding far fewer parameters. This model requires less memory and is friendly to on-chip devices.

Inspec keywords: feature extraction; convolution; image reconstruction; neural nets; learning (artificial intelligence); image resolution

Other keywords: image feature extraction; image reconstruction; novel SISR method; single image super-resolution methods; deeply recursive CNN; state-of-the-art methods; network structure; recursive learning; skip connection; parallelised 1 × 1 CNNs; accurate image super-resolution; on-chip devices; deep convolutional neural network

Subjects: Optical, image and video signal processing; Computer vision and image processing techniques; Neural computing techniques; Knowledge engineering techniques

References

    1. 1)
      • 27. Kingma, D., Ba, J.: ‘Adam: A method for stochastic optimization’. arXiv: 1412.6980, January 2017.
    2. 2)
      • 1. Duchon, C.E.: ‘Lanczos filtering in one and two dimensions’, J. Appl. Meteorol., 1979, 18, (8), pp. 1016–1022.
    3. 3)
      • 15. Tai, Y., Yang, J., Liu, X.: ‘Image super-resolution via deep recursive residual network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, US, July 2017, pp. 2790–2798.
    4. 4)
      • 7. Timofte, R., Smet, V.D., Gool, L.V.: ‘A + : adjusted anchored neighborhood regression for fast super-resolution’. Proc. Asian Conf. on Computer Vision, Singapore, Singapore, November 2014, pp. 111–126.
    5. 5)
      • 26. Hinton, G.E., Srivastava, N., Krizhevsky, A., et al: ‘Improving neural networks by preventing co-adaptation of feature detectors’, Comput. Sci., 2012, 3, (4), pp. 212–223.
    6. 6)
      • 25. He, K., Zhang, X., Ren, S., et al: ‘Delving deep into rectifiers: surpassing human-level performance on ImageNet classification’. Proc. IEEE Int. Conf. on Computer Vision, Santiago, Chile, December 2015, pp. 1026–1034.
    7. 7)
      • 9. Dong, C., Loy, C.C., He, K., et al: ‘Image super-resolution using deep convolutional networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 295–307.
    8. 8)
      • 4. Yang, J., Wright, J., Huang, T.S., et al: ‘Image super-resolution via sparse representation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 2861–2873.
    9. 9)
      • 22. Martin, D., Fowlkes, C., Tal, D., et al: ‘A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics’. Proc. IEEE Int. Conf. on Computer Vision, Vancouver, Canada, July 2001, pp. 416–423.
    10. 10)
      • 3. Bevilacqua, M., Roumy, A., Guillemot, C., et al: ‘Low-Complexity single image super-resolution based on nonnegative neighbor embedding’. Proc. British Machine Vision Conf., Surrey, Guildford, England, September 2012, pp. 135.1–135.10.
    11. 11)
      • 8. Schulter, S., Leistner, C., Bischof, H.: ‘Fast and accurate image upscaling with super-resolution forests’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, June 2015, pp. 3791–3799.
    12. 12)
      • 18. Mao, X.J., Shen, C., Yang, Y.B.: ‘Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections’. Proc. Adv. Neural Information Processing Systems, Barcelona, Spain, December 2016, pp. 2802–2810.
    13. 13)
      • 17. Chen, Z., Tong, Y.: ‘Face super-resolution through Wasserstein GANs’. arXiv: 1705.02438, May 2017.
    14. 14)
      • 2. Chang, H., Yeung, D.Y., Xiong, Y.: ‘Super-Resolution through neighbor embedding’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Washington, DC, June/July 2004, pp. 275–282.
    15. 15)
      • 13. Kim, J., Lee, J.K., Lee, K.M.: ‘Deeply-Recursive convolutional network for image super-resolution’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1637–1645.
    16. 16)
      • 11. Dong, C., Chen, C.L., Tang, X.: ‘Accelerating the super-resolution convolutional neural network’. Proc. European Conf. on Computer Vision, Amsterdam, Netherlands, October 2016, pp. 391–407.
    17. 17)
      • 20. Szegedy, C., Liu, W., Jia, Y., et al: ‘Going deeper with convolutions’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, June 2015, pp. 1–9.
    18. 18)
      • 14. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 770–778.
    19. 19)
      • 21. Shi, W., Caballero, J., Huszár, F., et al: ‘Real-Time single image and video super-resolution using an efficient sub-pixel convolutional neural network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1874–1883.
    20. 20)
      • 10. Dong, C., Chen, C.L., He, K., et al: ‘Learning a deep convolutional network for image super-resolution’. Proc. European Conf. on Computer Vision, Zurich, Switzerland, September 2014, pp. 184–199.
    21. 21)
      • 19. Lin, M., Chen, Q., Yan, S.: ‘Network in network’. arXiv: 1312.4400, March 2014.
    22. 22)
      • 16. Ledig, C., Theis, L., Huszar, F., et al: ‘Photo-Realistic single image super-resolution using a generative adversarial network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Honolulu, Hawaii, July 2017, pp. 105–114.
    23. 23)
      • 5. Roman, Z., Michael, E., Matan, P.: ‘On single image scale-up using sparse-representations’. Proc. Int. Conf. Curves and Surfaces, Avignon, France, June 2010, pp. 711–730.
    24. 24)
      • 23. Timofte, R., Rothe, R., Gool, L.V.: ‘Seven ways to improve example-based single image super resolution’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1865–1873.
    25. 25)
      • 6. Timofte, R., Smet, V.D., Gool, L.V.: ‘Anchored neighborhood regression for fast example-based super-resolution’. Proc. IEEE Int. Conf. on Computer Vision, Sydney, Australia, December 2013, pp. 1920–1927.
    26. 26)
      • 12. Kim, J., Lee, J.K., Lee, K.M.: ‘Accurate image super-resolution using very deep convolutional networks’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Seattle, WA, June 2016, pp. 1646–1654.
    27. 27)
      • 24. Arbeláez, P., Maire, M., Fowlkes, C., et al: ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (5), pp. 898–916.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.5907
Loading

Related content

content/journals/10.1049/iet-ipr.2018.5907
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address