Single-image super-resolution with total generalised variation and Shearlet regularisations
- Author(s): Wensen Feng 1 and Hong Lei 2
-
-
View affiliations
-
Affiliations:
1:
School of Automation and Electrical Engineering, University of Science and Technology, Beijing 100083, People's Republic of China;
2: Institute of Electronics, Chinese Academy of Science, Beijing 100190, People's Republic of China
-
Affiliations:
1:
School of Automation and Electrical Engineering, University of Science and Technology, Beijing 100083, People's Republic of China;
- Source:
Volume 8, Issue 12,
December 2014,
p.
833 – 845
DOI: 10.1049/iet-ipr.2013.0503 , Print ISSN 1751-9659, Online ISSN 1751-9667
In this study, the authors proposed a novel regularisation model for resolution enhancement of clean or noisy single image based on the total generalised variation (TGV) and Shearlet transform. The proposed model has two main contributions. Firstly, different from models with total variation regularisation, which assume that images consist of piecewise-constant areas, the author's TGV-based model is aware of higher-order smoothness, thus eliminates the staircase-like artefacts. Secondly, various image features including edges and fine details can be preserved by their model. This is nature since the Shearlets mathematically provide an optimally sparse approximation for the class of piecewise-smooth functions with rich geometric information. Moreover, to solve the proposed model, an efficient numerical scheme is explicitly developed based on the Nesterov's algorithm. A series of numerical experiments validate the effectiveness of the proposed method.
Inspec keywords: approximation theory; image resolution; piecewise constant techniques; image enhancement; transforms
Other keywords: staircase-like artefacts; total variation regularisation; geometric information; Shearlet transform; total generalised variation; resolution enhancement; piecewise-smooth functions; clean single image; higher-order smoothness; optimally sparse approximation; TGV-based model; Shearlet regularisations; Nesterov algorithm; noisy single image; single-image super-resolution; piecewise-constant areas
Subjects: Integral transforms in numerical analysis; Optical, image and video signal processing; Interpolation and function approximation (numerical analysis); Computer vision and image processing techniques; Interpolation and function approximation (numerical analysis); Integral transforms in numerical analysis
References
-
-
1)
-
48. Bredies, K.: ‘Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty’, Tech. Rep., Graz University of Technology, Graz, 2012.
-
-
2)
-
49. Knoll, F., Knoll, K., Pock, T., Stollberger, R.: ‘Second order total generalized variation (TGV) for MRI’, Mag. Res. Med., 2011, 65, (22), pp. 480–491 (doi: 10.1002/mrm.22595).
-
-
3)
- H. Takeda , S. Farsiu , P. Milanfar . Kernel regression for image processing and reconstruction. IEEE Trans. Image Process. , 2 , 349 - 366
-
4)
-
36. Chan, T., Esedoglu, S., Park, F., Yip, A.: ‘Recent developments in total variation image restoration’, in Paragios, N., Chen, Y., Faugeras, O. (Eds.): ‘Handbook of mathematical models in computer vision’ (Springer-Verlag, New York, USA, 2004).
-
-
5)
- A. Roussos , P. Maragos . Reversible interpolation of vectorial images by an anisotropic diffusion-projection PDE. Int. J. Comput. Vis. , 130 - 145
-
6)
-
19. Chang, H., Yeung, D.Y., Xiong, Y.: ‘Super-resolution through neighbor embedding’. Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2004, pp. 275–282.
-
-
7)
-
18. Zeyde, R., Elad, M., Protter, M.: ‘On single image scale-up using sparse-representations’. Proc. Int. Conf. Curves Surfaces, 2010, pp. 711–730.
-
-
8)
-
37. Saito, T., Komatsu, T.: ‘Super-resolution sharpening-demosaicking with spatially adaptive total-variation image regularization’. Proc. Pacific Rim Conf. Multimedia, 2005, pp. 246–256.
-
-
9)
-
47. Nesterov, Y.: ‘Smooth minimization of non-smooth functions’, Math. Program. A, 2005, 1, (103), pp. 127–152 (doi: 10.1007/s10107-004-0552-5).
-
-
10)
- L. Rudin , S. Osher , E. Fatemi . Nonlinear total variation based noise removal algorithms. Physica D , 259 - 268
-
11)
-
13. Luong, H.Q., Ledda, A., Philips, W.: ‘Non-local image interpolation’. Proc. IEEE Int. Conf. on Image Processing, 2006, pp. 693–696.
-
-
12)
-
7. Protter, M., Elad, M., Takeda, H., Milanfar, P.: ‘Generalizing the nonlocal-means to super-resolution reconstruction’, IEEE Trans. Image Process., 2009, 18, (1), pp. 36–51 (doi: 10.1109/TIP.2008.2008067).
-
-
13)
-
21. Becker, S., Bobin, J., Candès, E.J.: ‘NESTA: a fast and accurate first-order method for sparse recovery’, SIAM J. Imag. Sci., 2011, 4, pp. 1–39 (doi: 10.1137/090756855).
-
-
14)
-
39. Pulak, P., Bhabatosh, C.: ‘Super resolution image reconstruction through Bregman iteration using morphologic regularization’, IEEE Trans. Image Process., 2012, 21, (9), pp. 4029–4039 (doi: 10.1109/TIP.2012.2201492).
-
-
15)
-
48. Bredies, K.: ‘Recovering piecewise smooth multichannel images by minimization of convex functionals with total generalized variation penalty’, Tech. Rep., Graz University of Technology, Graz, 2012.
-
-
16)
-
46. Guo, W., Qin, J., Yin, Q.: ‘A new detail-preserving regularity scheme’, Rice CAAM Tech. Rep., Rice University, USA, 2013.
-
-
17)
-
9. Belahmidi, A., Guichard, F.: ‘A partial differential equation approach to image zoom’. Proc. IEEE Int. Conf. on Image Processing, 2004, pp. 649–652.
-
-
18)
- S. Farsiu , D. Robinson , M. Elad , P. Milanfar . Fast and robust multi-frame super-resolution. IEEE Trans. Image Process. , 10 , 1327 - 1344
-
19)
- A. Chambolle , P.L. Lions . Image recovery via total variation minimization and related problems. Numer. Math. , 2 , 167 - 188
-
20)
-
8. Getreuer, P.: ‘Contour stencils: total variation along curves for adaptive image interpolation’, SIAM J. Imaging Sci., 2011, 4, (3), pp. 954–979 (doi: 10.1137/100802785).
-
-
21)
- W.T. Freeman , T.R. Jones , E.C. Pasztor . Example-based super-resolution. IEEE Comput. Graph. Appl. , 2 , 56 - 65
-
22)
-
43. Benning, M., Brune, C., Burger, M., Muller, J.: ‘Higher-order TV methods-enhancement via Bregman iteration’, J. Sci. Comput. Manuscr., 2013, 54, (2–3), pp. 269–310 (doi: 10.1007/s10915-012-9650-3).
-
-
23)
-
21. Kim, K.I., Kwon, Y.: ‘Single-image super-resolution using sparse regression and natural image prior’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32, (6), pp. 1127–1133 (doi: 10.1109/TPAMI.2010.25).
-
-
24)
-
44. Muller, D., Lim, W.Q., Kutyniok, G., Weiss, G.: ‘Sparse multidi-mensional representation using Shearlets’. Proc. SPIE, Wavelet Applications in Signal and Image Processing XI, 2005, 5914, pp. 254–262.
-
-
25)
-
26. Yang, S., Wang, M., Chen, Y., Sun, Y.: ‘Single-image super-resolution reconstruction via learned geometric dictionaries and clustered sparse coding’, IEEE Trans. Image Process., 2012, 21, (9), pp. 4016–4028 (doi: 10.1109/TIP.2012.2201491).
-
-
26)
-
3. Park, S.C., Park, M.K., Kang, M.G.: ‘Super-resolution image reconstruction: a technical overview’, IEEE Signal Process. Mag., 2003, 20, (3), pp. 21–36 (doi: 10.1109/MSP.2003.1203207).
-
-
27)
- Z. Wang , A.C. Bovik , H.R. Sheikh , E.P. Simoncelli . Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. , 4 , 600 - 613
-
28)
-
20. Zhang, K., Gao, X., Li, X., Tao, D.: ‘Partially supervised neighbor embedding for example-based image super-resolution’, IEEE J. Sel. Top. Signal Process., 2011, 5, (2), pp. 230–239 (doi: 10.1109/JSTSP.2010.2048606).
-
-
29)
- G. Easley , D. Labate , W.Q. Lim . Sparse directional image representations using the discrete shearlet transform. Appl. Comput. Harmon. Anal. , 25 - 46
-
30)
-
12. Glasner, D., Bagon, S., Irani, M.: ‘Super-resolution from a single image’. Proc. IEEE Int. Conf. on Comput. Vis., 2009, pp. 349–356.
-
-
31)
-
7. Yang, J., Wang, Z., Lin, Z., Cohen, S., Huang, T.S.: ‘Couple dictionary training for image super-resolution’, IEEE Trans. Image Process., 2012, 21, (8), pp. 3467–3478 (doi: 10.1109/TIP.2012.2192127).
-
-
32)
-
38. Komatsu, Q., Zhang, L., Shen, H.: ‘Regional spatially adaptive total variation super-resolution with spatial information filtering and clustering’, IEEE Trans. Image Process., 2013, 22, (6), pp. 2327–2342 (doi: 10.1109/TIP.2013.2251648).
-
-
33)
-
7. Li, X., Hu, Y., Gao, X., Tao, D., Ning, B.: ‘A multi-frame image super-resolution method’, Signal Process., 2010, 90, (2), pp. 405–414 (doi: 10.1016/j.sigpro.2009.05.028).
-
-
34)
-
11. Tang, Y., Yan, P., Yuan, Y., Li, X.: ‘Single-image super-resolution via local learning’, Int. J. Mach. Learn. Cybern., 2011, 2, (1), pp. 15–23 (doi: 10.1007/s13042-011-0011-6).
-
-
35)
-
4. Allebach, J.P., Wong, P.W.: ‘Edge-directed interpolation’. Proc. IEEE Int. Conf. on Image Processing, 1996, pp. 707–710.
-
-
36)
- K. Jensen , D. Anastassiou . Subpixel edge localization and the interpolation of still images. IEEE Trans. Image Process. , 285 - 295
-
37)
-
41. Bredies, K., Kunisch, K., Pock, T.: ‘Total generalized variation’, SIAM J. Imaging Sci., 2010, 3, (3), pp. 492–526 (doi: 10.1137/090769521).
-
-
38)
-
30. Zhang, X., Lam, E.Y., Wu, E.X., Wong, K.K.: ‘Application of Tikhonov regularization to super-resolution reconstruction of brain MRI image’, Med. Image Inf., 2008, 49, (87), pp. 51–56 (doi: 10.1007/978-3-540-79490-5_8).
-
-
39)
- A. Maalouf , M.-C. Larabi . Colour image super-resolution using geometric grouplets. IET Image Process. , 2 , 168 - 180
-
40)
-
32. Ng, M., Shen, H., Lam, E., Zhang, L.: ‘A total variation regularization based super-resolution reconstruction algorithm for digital video’, EURASIP J. Adv. Signal Process., 2007, 2007, (074585), pp. 1–16 (doi: 10.1155/2007/74585).
-
-
41)
-
15. Marquina, A., Osher, S.J.: ‘Image super-resolution by TV-regularization and Bregman iteration’, J. Sci. Comput., 2008, 37, (3), pp. 367–382 (doi: 10.1007/s10915-008-9214-8).
-
-
42)
-
22. Tang, Y., Yuan, Y., Yan, P., Li, X.: ‘Single-image super-resolution via sparse coding regression’. Proc. IEEE Int. Conf. Image Graphics, 2011, pp. 267–272.
-
-
43)
-
40. Dong, W., Zhang, L., Shi, G., Wu, X.: ‘Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization’, IEEE Trans. Image Process., 2011, 20, (7), pp. 1838–1857 (doi: 10.1109/TIP.2011.2108306).
-
-
44)
-
10. Gao, X., Zhang, K., Tao, D., Li, X.: ‘Image super-resolution with sparse neighbor embedding’, IEEE Trans. Image Process., 2012, 21, (7), pp. 3194–3205 (doi: 10.1109/TIP.2012.2190080).
-
-
45)
-
51. Chambolle, A., Pock, T.: ‘A first-order primal-dual algorithm for convex problems with applications to Imaging’, J. Math. Imag. Vis., 2011, 40, (1), pp. 120–145 (doi: 10.1007/s10851-010-0251-1).
-
-
46)
- J. Yang , J. Wright , T. Huang , Y. Ma . Image super-resolution via sparse representation. IEEE Trans. Image Process. , 11 , 2861 - 2873
-
47)
-
29. Mallat, S., Guo, Y.: ‘Super-resolution with sparse mixing estimators’, IEEE Trans. Image Process., 2010, 19, (11), pp. 2889–2900 (doi: 10.1109/TIP.2010.2049927).
-
-
48)
-
11. Fisher, Y.: ‘Fractal image compression: theory and application to digital images’ (Springer-Verlag Press, New York, 1995).
-
-
49)
-
28. Yu, G., Sapiro, G., Mallat, S.: ‘Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity’, IEEE Trans. Image Process., 2012, 21, (5), pp. 2481–2499 (doi: 10.1109/TIP.2011.2176743).
-
-
50)
-
25. Gao, X., Zhang, K., Li, X., Tao, D.: ‘Joint learning for single-image super-resolution via a coupled constraint’, IEEE Trans. Image Process., 2012, 21, (2), pp. 469–480 (doi: 10.1109/TIP.2011.2161482).
-
-
51)
- Q. Wang , R.K. Ward . A new orientation-adaptive interpolation method. IEEE Trans. Image Process. , 889 - 900
-
52)
-
2. Dyn, N., Levin, D., Rippa, S.: ‘Data dependent triangulations for piecewise linear interpolation’, IMA J. Numer. Anal., 1990, 10, (1), pp. 137–154 (doi: 10.1093/imanum/10.1.137).
-
-
1)