access icon free Maximised self-similarity upsampler

Image self-similarity property is important to super-resolution reconstruction. However, how to effectively exploit the self-similarity information to reconstruct an underlying high-resolution image is still a challenging problem. The authors propose a novel model for solving the single image upsampling problem with the self-similarity property. First, the authors construct a statistical prior that requires maximising the similarity between the low- and high-resolution image pairs. Then, the authors develop an alternative Gaussian approximation solver based on the Gaussian mixture model to find the optimal high-resolution output. To obtain a better performance, the authors summarise some refined implementation skills to raise the reconstruction quality. For demonstration, a series of objective and subjective measurements are used to evaluate the performance of the model.

Inspec keywords: Gaussian processes; optimisation; image resolution; approximation theory; image reconstruction

Other keywords: Gaussian mixture model; self-similarity property; maximised self-similarity upsampler; super resolution reconstruction; image upsampling problem; high-resolution image; image self-similarity property; Gaussian approximation solver

Subjects: Interpolation and function approximation (numerical analysis); Interpolation and function approximation (numerical analysis); Optimisation techniques; Optimisation techniques; Other topics in statistics; Computer vision and image processing techniques; Optical, image and video signal processing; Other topics in statistics

References

    1. 1)
      • 26. Lu, X., Yuan, Y., Yan, P.: ‘Image super-resolution via double sparsity regularized manifold learning’, IEEE Trans. Circuits Syst. Video Technol., 2013, 23, (12), pp. 20222033.
    2. 2)
      • 23. Chang, H., Yeung, D., Xiong, Y.: ‘Super-resolution through neighbor embedding’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2004), Washington, DC, United States, 2004, vol. 1, pp. 275282.
    3. 3)
      • 48. HaCohen, Y., Fattal, R., Lischinski, D.: ‘Image upsampling via texture hallucination’. IEEE Int. Conf. Computational Photography (ICCP 2010), Cambridge, United States, 2010, pp. 18.
    4. 4)
      • 51. Yang, J., Lin, Z., Cohen, S.: ‘Fast image super-resolution based on in-place example regression’. IEEE Conf. Computer Vision and Pattern Recognition, 2013, pp. 10591066.
    5. 5)
      • 46. Zhu, Z., Guo, F., Yu, H., et al: ‘Fast single image super-resolution via self-example learning and sparse representation’, IEEE Trans. Multimed., 2014, 16, (8), pp. 21782190.
    6. 6)
      • 27. Liang, M., Du, J., Cao, S., et al: ‘Super-resolution reconstruction based on multisource bidirectional similarity and non-local similarity matching’, IET Image Process., 2015, 9, (11), pp. 931942.
    7. 7)
      • 14. Zhang, X., Liu, Q., Li, X., et al: ‘Non-local feature back-projection for image super-resolution’, IET Image Process., 2016, 10, (5), pp. 398408.
    8. 8)
      • 36. Zeyde, R., Elad, M., Protter, M.: ‘On single image scale-up using sparse-representations’. 7th Int. Conf. Curves and Surfaces, Avignon, France, 2010, vol. 6920, pp. 711730.
    9. 9)
      • 18. He, H., Siu, W.: ‘Single image super-resolution using Gaussian process regression’. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 449456.
    10. 10)
      • 3. Yang, J., Wright, J., Huang, T., et al: ‘Image super-resolution via sparse representation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 28612873.
    11. 11)
      • 9. Tai, Y., Tong, W., Tang, C.: ‘Perceptually-inspired and edge-directed color image super-resolution’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2006), 2006, vol. 2, pp. 19481955.
    12. 12)
      • 40. Yang, S., Wang, M., Chen, Y., et al: ‘Single-image super-resolution reconstruction via learned geometric dictionaries and clustered sparse coding’, IEEE Trans. Image Process., 2012, 21, (9), pp. 40164028.
    13. 13)
      • 44. Dong, W., Zhang, L., Lukac, R., et al: ‘Sparse representation based image interpolation with nonlocal autoregressive modeling’, IEEE Trans. Image Process., 2013, 22, (4), pp. 13821394.
    14. 14)
      • 10. Li, M., Nguyen, T.: ‘Markov random field model-based edge-directed image interpolation’, IEEE Trans. Image Process., 2008, 17, (7), pp. 11211128.
    15. 15)
      • 50. Dong, W., Zhang, L., Shi, G., et al: ‘Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization’, IEEE Trans. Image Process., 2011, 20, (7), pp. 18381857.
    16. 16)
      • 29. Fattal, R.: ‘Image upsampling via imposed edge statistics’. ACM SIGGRAPH, California, 2007, 26, (3), pp. 95.
    17. 17)
      • 32. Sun, J., Sun, J., Xu, Z., et al: ‘Image super-resolution using gradient profile prior’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2008), Anchorage, AK, 2008, pp. 18.
    18. 18)
      • 31. Shan, Q., Li, Z., Jia, J., et al: ‘Fast image/video upsampling’. ACM SIGGRAPH Asia, 2008, pp. 17.
    19. 19)
      • 12. Xu, H., Zhai, G., Yang, X.: ‘Single image super-resolution with detail enhancement based on local fractal analysis of gradient’, IEEE Trans. Circuits Syst. Video Technol., 2013, 23, (10), pp. 17401754.
    20. 20)
      • 2. Freedman, G., Fattal, R.: ‘Image and video upscaling from local self-examples’, ACM Trans. Graph., 2011, 30, (2), pp. 110.
    21. 21)
      • 53. Yu, J., Gao, X., Tao, D., et al: ‘A unified learning framework for single image super-resolution’, IEEE Trans. Neural Netw. Learn. Syst., 2014, 25, (4), pp. 780792.
    22. 22)
      • 43. Ren, J., Liu, J., Guo, Z.: ‘Context-aware sparse decomposition for image denoising and super-resolution’, IEEE Trans. Image Process., 2013, 22, (4), pp. 14561469.
    23. 23)
      • 52. Zhang, K., Gao, X., Tao, D., et al: ‘Single image super-resolution with multiscale similarity learning’, IEEE Trans. Neural Netw. Learn. Syst., 2013, 24, (10), pp. 16481659.
    24. 24)
      • 5. Li, X., Orchard, M.: ‘New edge-directed interpolation’, IEEE Trans. Image Process., 2001, 10, (10), pp. 15211527.
    25. 25)
      • 4. Hou, H., Andrews, H.: ‘Cubic splines for image interpolation and digital filtering’, IEEE Trans. Acoust. Speech Signal Process., 1978, 26, (6), pp. 508517.
    26. 26)
      • 45. Peleg, T., Elad, M.: ‘A statistical prediction model based on sparse representations for single image super-resolution’, IEEE Trans. Image Process., 2014, 23, (6), pp. 25692582.
    27. 27)
      • 39. Zhang, H., Zhang, Y., Huang, T.: ‘Efficient sparse representation based image super resolution via dual dictionary learning’. IEEE Int. Conf. Multimedia and Expo, 2011, pp. 16.
    28. 28)
      • 1. Glasner, D., Bagon, S., Irani, M.: ‘Super-resolution from a single image’. 12th Int. Conf. Computer Vision, 2009, pp. 349356.
    29. 29)
      • 8. Zhang, L., Wu, X.: ‘An edge-guided image interpolation algorithm via directional filtering and data fusion’, IEEE Trans. Image Process., 2006, 15, (8), pp. 22262238.
    30. 30)
      • 15. Freeman, W., Jones, T., Pasztor, E.: ‘Example-based super-resolution’, IEEE Comput. Graph. Appl., 2002, 22, (2), pp. 5665.
    31. 31)
      • 30. Dai, S., Han, M., Xu, W., et al: ‘Soft edge smoothness prior for alpha channel super resolution’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2007), Minneapolis, MN, United States, 2007, pp. 18.
    32. 32)
      • 13. Li, M., Liu, J., Ren, J., et al: ‘Adaptive general scale interpolation based on weighted autoregressive models’, IEEE Trans. Circuits Syst. Video Technol., 2015, 25, (2), pp. 200211.
    33. 33)
      • 38. Mallat, S., Yu, G.: ‘Super-resolution with sparse mixing estimators’, IEEE Trans. Image Process., 2010, 19, (11), pp. 28892900.
    34. 34)
      • 28. Baker, S., Kanade, T.: ‘Limits on super-resolution and how to break them’, IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24, (9), pp. 11671183.
    35. 35)
      • 41. Yang, J., Wang, Z., Lin, Z., et al: ‘Coupled dictionary training for image super-resolution’, IEEE Trans. Image Process., 2012, 21, (8), pp. 34673478.
    36. 36)
      • 6. Asuni, N., Giachetti, A.: ‘Accuracy improvements and artifacts removal in edge based image interpolation’. VISAPP 2008, 2008, pp. 18.
    37. 37)
      • 42. Wang, S., Zhang, L., Liang, Y., et al: ‘Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2012), 2012, pp. 22162223.
    38. 38)
      • 37. Kim, K., Kwon, Y.: ‘Single-image super-resolution using sparse regression and natural image prior’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32, (6), pp. 11271133.
    39. 39)
      • 7. Giachetti, A., Asuni, N.: ‘Real-time artifact-free image upscaling’, IEEE Trans. Image Process., 2011, 20, (10), pp. 27602768.
    40. 40)
      • 16. Lin, Z., He, J., Tang, X., et al: ‘Limits of learning-based superresolution algorithms’. 11th IEEE Int. Conf. Computer Vision (ICCV 2007), Rio de Janeiro, Brazil, 2007, pp. 18.
    41. 41)
      • 24. Gao, X., Zhang, K., Tao, D., et al: ‘Joint learning for single-image super-resolution via a coupled constraint’, IEEE Trans. Image Process., 2012, 21, (2), pp. 469480.
    42. 42)
      • 33. Tai, Y., Liu, S., Brown, M., et al: ‘Super resolution using edge prior and single image detail synthesis’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR 2010), San Francisco, United States, 2010, pp. 24002407.
    43. 43)
      • 17. Kim, K., Kwon, Y.: ‘Example-based learning for single-image super-resolution’. DAGM, 2008, pp. 456465.
    44. 44)
      • 22. Yang, M., Wang, Y.: ‘A self-learning approach to single image super-resolution’, IEEE Trans. Multimed., 2013, 15, (3), pp. 498508.
    45. 45)
      • 11. Wang, L., Xiang, S., Meng, G., et al: ‘Edge-directed single-image super-resolution via adaptive gradient magnitude self-interpolation’, IEEE Trans. Circuits Syst. Video Technol., 2013, 23, (8), pp. 12891299.
    46. 46)
      • 25. Jiang, J., Hu, R., Han, Z., et al: ‘Efficient single image super-resolution via graph embedding’. IEEE Int. Conf. Multimedia and Expo (ICME 2012), Melbourne, VIC, 2012, pp. 610615.
    47. 47)
      • 49. Zontak, M., Irani, M.: ‘Internal statistics of a single natural image’. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 977984.
    48. 48)
      • 19. Zhang, K., Gao, X., Tao, D., et al: ‘Single image super-resolution with non-local means and steering kernel regression’, IEEE Trans. Image Process., 2012, 21, (11), pp. 45444556.
    49. 49)
      • 21. Dong, C., Loy, C., He, K., et al: ‘Learning a deep convolutional network for image super-resolution’. European Conf. Computer Vision (ECCV 2014), 2014, pp. 184199.
    50. 50)
      • 35. Zhang, H., Zhang, Y., Li, H., et al: ‘Generative Bayesian image super resolution with natural image prior’, IEEE Trans. Image Process., 2012, 21, (9), pp. 40544067.
    51. 51)
      • 20. Xiong, Z., Xu, D., Sun, X., et al: ‘Example-based super-resolution with soft information and decision’, IEEE Trans. Multimed., 2013, 15, (6), pp. 14581465.
    52. 52)
      • 47. Cheng, M., Wang, C., Li, J.: ‘Single-image super-resolution in RGB space via group sparse representation’, IET Image Process., 2015, 9, (6), pp. 461467.
    53. 53)
      • 34. Babacan, S., Molina, R., Katsaggelos, A.: ‘Variational Bayesian super resolution’, IEEE Trans. Image Process., 2011, 20, (4), pp. 984999.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2017.0274
Loading

Related content

content/journals/10.1049/iet-ipr.2017.0274
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading