access icon free Multiframe super-resolution based on a high-order spatially weighted regularisation

Here, the authors propose a spatially weighted super-resolution (SR) algorithm, which takes into consideration the distribution of every information that characterise different image areas. The authors investigate to use a combined spatially weighted regularisation of the bilateral total variation and a second-order term increasing then the robustness of the proposed SR approach with respect to blur and noise degradations. In addition, the authors propose an iterative Bregman iteration algorithm to resolve the obtained optimisation SR problem. As a result, this regularisation is more efficient and easier to implement; moreover, it preserves well the smooth regions of the image and also sharp edges. Using different simulated and real tests, the authors prove the efficiency of the proposed algorithm compared to some SR methods.

Inspec keywords: iterative methods; image resolution; optimisation

Other keywords: high-order spatially weighted regularisation; sharp edges; optimisation SR problem; SR algorithm; smooth image regions; iterative Bregman iteration algorithm; blur degradations; multiframe super-resolution; noise degradations; second-order term; bilateral total variation

Subjects: Interpolation and function approximation (numerical analysis); Optimisation techniques; Optical, image and video signal processing; Computer vision and image processing techniques; Optimisation techniques; Interpolation and function approximation (numerical analysis)

References

    1. 1)
      • 24. Zhang, X., Lam, E.Y., Wu, E.X., et al: ‘Application of Tikhonov regularization to super-resolution reconstruction of brain MRI images’, in ‘Medical imaging and informatics’ (Springer, Berlin, 2008), pp. 5156.
    2. 2)
      • 50. Li, L., Xie, Y., Hu, W., et al: ‘Single image super-resolution using combined total variation regularization by split Bregman iteration’, Neurocomputing, 2014, 142, pp. 551560.
    3. 3)
      • 49. Purkait, P., Chanda, B.: ‘Super resolution image reconstruction through Bregman iteration using morphologic regularization’, IEEE Trans. Image Process., 2012, 21, (9), pp. 40294039.
    4. 4)
      • 6. Peleg, T., Elad, M.: ‘A statistical prediction model based on sparse representations for single image super-resolution’, IEEE Trans. Image Process., 2014, 23, (6), pp. 25692582.
    5. 5)
      • 25. Rudin, L.I., Osher, S., Fatemi, E.: ‘Nonlinear total variation based noise removal algorithms’, Physica D, 1992, 60, (1–4), pp. 259268.
    6. 6)
      • 7. Feng, W., Lei, H.: ‘Single-image super-resolution with total generalised variation and shearlet regularisations’, IET Image Process., 2014, 8, (12), pp. 833845.
    7. 7)
      • 30. Laghrib, A., Ghazdali, A., Hakim, A., et al: ‘A multi-frame super-resolution using diffusion registration and a nonlocal variational image restoration’, Comput. Math. Appl., 2016, 72, (9), pp. 25352548.
    8. 8)
      • 41. Ren, Z., He, C., Zhang, Q.: ‘Fractional order total variation regularization for image super-resolution’, Signal Process., 2013, 93, (9), pp. 24082421.
    9. 9)
      • 26. Ng, M.K., Shen, H., Lam, E.Y., et al: ‘A total variation regularization based super-resolution reconstruction algorithm for digital video’, EURASIP J. Adv. Signal Process..
    10. 10)
      • 9. Cheng, M., Wang, C., Li, J.: ‘Single-image super-resolution in RGB space via group sparse representation’, IET Image Process., 2014, 9, (6), pp. 461467.
    11. 11)
      • 22. Golub, G.H., Hansen, P.C., O'Leary, D.P.: ‘Tikhonov regularization and total least squares’, SIAM J. Matrix Anal. Appl., 1999, 21, (1), pp. 185194.
    12. 12)
      • 11. Protter, M., Elad, M., Takeda, H., et al: ‘Generalizing the nonlocal-means to super-resolution reconstruction’, IEEE Trans. Image Process., 2009, 18, (1), pp. 3651.
    13. 13)
      • 55. Anbarjafari, G., Izadpanahi, S., Demirel, H.: ‘Video resolution enhancement by using discrete and stationary wavelet transforms with illumination compensation’, Signal Image Video Process., 2015, 9, (1), pp. 8792.
    14. 14)
      • 58. Bredies, K., Kunisch, K., Pock, T.: ‘Total generalized variation’, SIAM J. Imaging Sci., 2010, 3, (3), pp. 492526.
    15. 15)
      • 35. Kasetkasem, T., Arora, M.K., Varshney, P.K.: ‘Super-resolution land cover mapping using a Markov random field based approach’, Remote Sens. Environ., 2005, 96, (3), pp. 302314.
    16. 16)
      • 12. Milanfar, P.: ‘Super-resolution imaging’ (CRC Press, 2010).
    17. 17)
      • 1. Tsai, R.Y.: ‘Multiframe image restoration and registration’, Adv. Comput. Vis. Image Process., 1984, 1, (2), pp. 317339.
    18. 18)
      • 40. Yuan, Q., Zhang, L., Shen, H.: ‘Multiframe super-resolution employing a spatially weighted total variation model’, IEEE Trans. Circuits Syst. Video Technol., 2012, 22, (3), pp. 379392.
    19. 19)
      • 21. Zhang, L., Zhang, H., Shen, H., et al: ‘A super-resolution reconstruction algorithm for surveillance images’, Signal Process., 2010, 90, (3), pp. 848859.
    20. 20)
      • 13. Kim, J., Kwon Lee, J., Mu Lee, K.: ‘Accurate image super-resolution using very deep convolutional networks’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, 2016, pp. 16461654.
    21. 21)
      • 8. Walha, R., Drira, F., Lebourgeois, F., et al: ‘Resolution enhancement of textual images: a survey of single image-based methods’, IET Image Process., 2016, 10, (4), pp. 325337.
    22. 22)
      • 34. Laghrib, A., Hakim, A., Raghay, S.: ‘An iterative image super-resolution approach based on Bregman distance’, Signal Process., Image Commun., 2017, 58, pp. 2434.
    23. 23)
      • 57. Hore, A., Ziou, D.: ‘Image quality metrics: PSNR vs. SSIM’. 2010 20th Int. Conf. on Pattern Recognition (ICPR), IEEE, 2010, pp. 23662369.
    24. 24)
      • 18. Cai, K., Shi, J., Xiong, S., et al: ‘Edge adaptive image resolution enhancement in video sensor network’, Opt. Commun., 2011, 284, (19), pp. 44464451.
    25. 25)
      • 14. Tian, J., Ma, K.-K.: ‘A survey on super-resolution imaging, signal’, Image Video Process., 2011, 5, (3), pp. 329342.
    26. 26)
      • 52. Zomet, A., Rav-Acha, A., Peleg, S.: ‘Robust super-resolution’. CVPR 2001. Proc. of the 2001 IEEE Computer Society Conf. onComputer Vision and Pattern Recognition, 2001mt, IEEE, vol. 1, 2001, p. I.
    27. 27)
      • 17. Greenspan, H., Oz, G., Kiryati, N., et al: ‘MRI inter-slice reconstruction using super-resolution’, Magn. Reson. Imaging, 2002, 20, (5), pp. 437446.
    28. 28)
      • 19. Zhang, K., Mu, G., Yuan, Y., et al: ‘Video super-resolution with 3d adaptive normalized convolution’, Neurocomputing, 2012, 94, pp. 140151.
    29. 29)
      • 27. Babacan, S.D., Molina, R., Katsaggelos, A.K.: ‘Total variation super resolution using a variational approach’. 2008 15th IEEE Int. Conf. on. Image Processing (ICIP), IEEE, 2008, pp. 641644.
    30. 30)
      • 51. Chen, Q., Montesinos, P., Sun, Q.S., et al: ‘Adaptive total variation denoising based on difference curvature’, Image Vis. Comput., 2010, 28, (3), pp. 298306.
    31. 31)
      • 29. Mochizuki, Y., Kameda, Y., Imiya, A., et al: ‘Variational method for super-resolution optical flow’, Signal Process., 2011, 91, (7), pp. 15351567.
    32. 32)
      • 23. Nguyen, N., Milanfar, P., Golub, G.: ‘A computationally efficient superresolution image reconstruction algorithm’, IEEE Trans. Image Process., 2001, 10, (4), pp. 573583.
    33. 33)
      • 20. Izadpanahi, S., Demirel, H.: ‘Motion based video super resolution using edge directed interpolation and complex wavelet transform’, Signal Process., 2013, 93, (7), pp. 20762086.
    34. 34)
      • 46. El Mourabit, I., El Rhabi, M., Hakim, A., et al: ‘A new denoising model for multi-frame super-resolution image reconstruction’, Signal Process., 2017, 132, pp. 5165.
    35. 35)
      • 53. Papafitsoros, K., Schönlieb, C.-B.: ‘A combined first and second order variational approach for image reconstruction’, J. Math. Imaging Vis., 2014, 48, (2), pp. 308338.
    36. 36)
      • 37. Chen, X., Qi, C.: ‘Document image super-resolution using structural similarity and Markov random field’, IET Image Process., 2014, 8, (12), pp. 687698.
    37. 37)
      • 33. Zeng, X., Yang, L.: ‘A robust multiframe super-resolution algorithm based on halfquadratic estimation with modified BTV regularization’, Digit. Signal Process., 2013, 23, (1), pp. 98109.
    38. 38)
      • 2. Chaudhuri, S.: ‘Super-resolution imaging’ (Springer Science & Business Media, 2001).
    39. 39)
      • 36. Ardila, J.P., Tolpekin, V.A., Bijker, W., et al: ‘Markov-random-field-based superresolution mapping for identification of urban trees in VHR images’, ISPRS J. Photogramm. Remote Sens., 2011, 66, (6), pp. 762775.
    40. 40)
      • 56. Ning, B., Gao, X.: ‘Multi-frame image super-resolution reconstruction using sparse co-occurrence prior and sub-pixel registration’, Neurocomputing, 2013, 117, pp. 128137.
    41. 41)
      • 54. Goldstein, T., Osher, S.: ‘The split Bregman method for l1-regularized problems’, SIAM J. Imaging Sci., 2009, 2, (2), pp. 323343.
    42. 42)
      • 31. Patanavijit, V., Jitapunkul, S.: ‘An iterative super-resolution reconstruction of image sequences using fast affine block-based registration with BTV regularization’. APCCAS 2006. IEEE Asia Pacific Conf. on Circuits and Systems, 2006, IEEE, 2006, pp. 17171720.
    43. 43)
      • 45. Maiseli, B.: ‘Diffusion-steered super-resolution method based on the Papoulis–Gerchberg algorithm’, IET Image Process., 2016, 10, (10), pp. 683692.
    44. 44)
      • 4. Laghrib, A., Ezzaki, M., El Rhabi, M., et al: ‘Simultaneous deconvolution and denoising using a second order variational approach applied to image super resolution’, Comput. Vis. Image Underst..
    45. 45)
      • 5. Yang, J., Wright, J., Huang, T.S., et al: ‘Image super-resolution via sparse representation’, IEEE Trans. Image Process., 2010, 19, (11), pp. 28612873.
    46. 46)
      • 28. Marquina, A., Osher, S.J.: ‘Image super-resolution by TV-regularization and Bregman iteration’, J. Sci. Comput., 2008, 37, (3), pp. 367382.
    47. 47)
      • 3. Park, S.C., Park, M.K., Kang, M.G.: ‘Super-resolution image reconstruction: a technical overview’, IEEE Signal Process. Mag., 2003, 20, (3), pp. 2136.
    48. 48)
      • 10. Farsiu, S., Robinson, M.D., Elad, M., et al: ‘Fast and robust multiframe super resolution’, IEEE Trans. Image Process., 2004, 13, (10), pp. 13271344.
    49. 49)
      • 42. Laghrib, A., Hakim, A., Raghay, S.: ‘A combined total variation and bilateral filter approach for image robust super resolution’, EURASIP J. Image Video Process., 2015, 2015, (1), pp. 110.
    50. 50)
      • 43. Zeng, W., Lu, X., Fei, S.: ‘Image super-resolution employing a spatial adaptive prior model’, Neurocomputing, 2015, 162, pp. 218233.
    51. 51)
      • 39. Villena, S., Vega, M., Babacan, S.D., et al: ‘Bayesian combination of sparse and non-sparse priors in image super resolution’, Digit. Signal Process., 2013, 23, (2), pp. 530541.
    52. 52)
      • 47. Chambolle, A., Pock, T.: ‘A first-order primal-dual algorithm for convex problems with applications to imaging’, J. Math. Imaging Vis., 2011, 40, (1), pp. 120145.
    53. 53)
      • 32. Li, X., Hu, Y., Gao, X., et al: ‘A multi-frame image super-resolution method’, Signal Process., 2010, 90, (2), pp. 405414.
    54. 54)
      • 38. Chen, Y.: ‘Higher-order MRFS based image super resolution: why not map?’, IET Image Process., 2016, 10, (4), pp. 297303.
    55. 55)
      • 44. Maiseli, B., Elisha, O., Mei, J., et al: ‘Edge preservation image enlargement and enhancement method based on the adaptive perona–malik non-linear diffusion model’, IET Image Process., 2014, 8, (12), pp. 753760.
    56. 56)
      • 15. Nasrollahi, K., Moeslund, T.B.: ‘Super-resolution: a comprehensive survey’, Mach. Vis. Appl., 2014, 25, (6), pp. 14231468.
    57. 57)
      • 16. Yue, L., Shen, H., Li, J., et al: ‘Image super-resolution: the techniques, applications, and future’, Signal Process., 2016, 128, pp. 389408.
    58. 58)
      • 48. Unger, M., Pock, T., Werlberger, M., et al: ‘A convex approach for variational super-resolution’. Joint Pattern Recognition Symp., Springer, 2010, pp. 313322.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2017.1046
Loading

Related content

content/journals/10.1049/iet-ipr.2017.1046
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading