access icon free Fractional-order tensor regularisation for image inpainting

© The Institution of Engineering and Technology
Received 01/12/2016, Accepted 28/05/2017, Revised 05/05/2017, Published 05/06/2017

Compared with classic integer-order calculus, fractional calculus is a more powerful mathematical method that non-linearly preserves and enhances image features in different frequency bands. In order to extend fractional-in-space diffusion scheme with matrix-valued diffusivity to perform superior image inpainting, the authors build the new fractional-order tensor regularisation (FTR) model by utilising the newly defined fractional-order structure tensor (FST) to control the regularisation process. The proposed model is derived as a process that minimises a functional proportional to the FST composed of the inner product of the fractional derivative vector and its transposition; hence, the new model not only inherits genuine anisotropism of tensor regularisation, but is also better equipped to handle subtle details and complex structures because of the characteristics of fractional calculus. To minimise the proposed functional, the corresponding Euler–Lagrange equation is deduced, and the anisotropism of the proposed model is analysed accordingly. Fractional-order derivative masks in positive x and y directions and negative x and y directions are implemented according to the shifted Grümwald–Letnikov definition, and a proper iterative numerical scheme is analysed. According to experimental results on various test images, the proposed FTR inpainting model demonstrates superior inpainting performance both in noiseless and noisy scenarios.

Inspec keywords: image restoration; image enhancement; tensors; iterative methods; vectors

Other keywords: noiseless scenarios; classic integer-order calculus; mathematical method; Euler-Lagrange equation; iterative numerical scheme; image feature enhancement; fractional calculus; complex structures; noisy scenarios; FTR model; fractional-order derivative masks; fractional derivative vector; fractional-in-space diffusion scheme; fractional-order tensor regularisation; image inpainting; frequency bands; fractional-order structure tensor; shifted Grümwald-Letnikov definition; FST; matrix-valued diffusivity

Subjects: Interpolation and function approximation (numerical analysis); Interpolation and function approximation (numerical analysis); Optical, image and video signal processing; Linear algebra (numerical analysis); Linear algebra (numerical analysis); Computer vision and image processing techniques

References

    1. 1)
      • 10. Li, P., Li, S.J., Yao, Z.-A., et al: ‘Two anisotropic fourth-order partial differential equations for image inpainting’, IET Image Process., 2013, 7, (3), pp. 260–269.
    2. 2)
      • 8. Marquina, A., Osher, S.-J: ‘Image super-resolution by TV-regularization and Bregman iteration’, J. Sci. Comput., 2008, 37, (3), pp. 367–382.
    3. 3)
      • 11. Chan, T.-F., Shen, J.: ‘Mathematical models for local non-texture inpainting’, SIAM J. Appl. Math., 2002, 62, (3), pp. 1019–1043.
    4. 4)
      • 9. Bertalmio, M., Sapiro, G., Caselles, V., et al: ‘Image inpainting’. Proc. Conf. Computer graphics and Interactive Techniques (SIGGRAPH 2000), New Orleans, Louisiana, July 2000, pp. 417–424.
    5. 5)
      • 42. Chambolle, A.: ‘An algorithm for total variation minimization and applications’, J. Math. Imaging Vis., 2004, 20, pp. 89–97.
    6. 6)
      • 4. Yastrebov, V., Veselov, A., Filippov, B.: ‘Analysis of exemplar based inpainting techniques for error concealment’. Proc. Int. Symp. Problems of Redundancy in Information and Control Systems, Saint-Petersburg, Russia, June 2014, pp. 103–106.
    7. 7)
      • 2. Criminisi, A., Pérez, P., Toyama, K.: ‘Region filling and object removal by exemplar-based image inpainting’, IEEE Trans. Image Process., 2004, 13, (9), pp. 1200–1212.
    8. 8)
      • 23. Estellers, V., Soatto, S., Bresson, X.: ‘Adaptive regularization with the structure tensor’, IEEE Trans. Image Process., 2015, 24, (6), pp. 1777–1790.
    9. 9)
      • 50. Lieu, L.-H., Vese, L.-A.: ‘Image restoration and decomposition via bounded total variation and negative Hilbert-Sobolev spaces’, Appl. Math. Optim., 2008, 58, (2), pp. 167–193.
    10. 10)
      • 7. Takuya, U., Kazushi, M., Akira, H.: ‘A method for single frame super resolution with inpainting based on sparse dictionary learning’. Annual Summit and Conf. Asia-Pacific Signal and Information Processing Association (APSIPA), Angkor Wat, Cambodia, December 2014, pp. 205–211.
    11. 11)
      • 41. Osher, S., Burger, M., Goldfarb, D., et al: ‘An iterative regularization method for total variation-based image restoration’, Multiscale Model. Simul., 2005, 4, (2), pp. 460–489.
    12. 12)
      • 14. Rudin, L., Osher, S., Fatemi, E.: ‘Nonlinear total variation based noise removal algorithms’, Physica D: Nonlinear Phenom., 1992, 60, (1–4), pp. 259–268.
    13. 13)
      • 19. Perona, P., Malik, J.: ‘Scale-space and edge detection using anisotropic diffusion’, IEEE Trans. Pattern Anal. Mach. Intell., 1990, 12, (7), pp. 629–639.
    14. 14)
      • 45. Li, B., Xie, W.: ‘Adaptive fractional differential approach and its application to medical image enhancement’, Comput. Electr. Eng., 2015, 45, pp. 324–335.
    15. 15)
      • 37. Hildebrandt, S., Kinderlehrer, D., Miranda, M.: ‘Calculus of variations and partial differential equations’ (Springer, 2000).
    16. 16)
      • 17. Ma, W., You, Y., Kaveh, M.: ‘Image restoration regularized by a fourth-order PDE’, Proc. Spie, 2009, 7245, (1), pp. 724509–724509-9.
    17. 17)
      • 39. Strang, G.: ‘Linear algebra and its applications’ (Brooks Cole, 2005, 4th edn.), ISBN 978-0-03-010567-8.
    18. 18)
      • 40. Meerschaert, M.-M., Tadjeran, C.: ‘Finite difference approximations for fractional advection-dispersion flow equations’, J. Comput. Appl. Math., 2004, 172, (1), pp. 65–77.
    19. 19)
      • 31. Zhang, J., Chen, K.: ‘A total fractional-order variation model for image restoration with non-homogeneous boundary conditions and its numerical solution’, SIAM J. Imaging Sci., 2015, 8, (4), pp. 2487–2518.
    20. 20)
      • 18. Wu, T., Yang, Y., Pang, Z.: ‘A modified fixed-point iterative algorithm for image restoration using fourth-order PDE model’, Appl. Numer. Math., 2012, 62, (2), pp. 79–90.
    21. 21)
      • 48. Tian, D., Xue, D., Wang, D.: ‘A fractional-order adaptive regularization primal-dual algorithm for image denoising’, Inf. Sci., 2015, 296, (7), pp. 147–159.
    22. 22)
      • 46. Yu, Q., Vegh, V., Liu, F., et al: ‘A variable order fractional differential-based texture enhancement algorithm with application in medical imaging’, Plos One, 2015, 10, (7), p. e0132952.
    23. 23)
      • 1. Karianakis, N., Maragos, P.: ‘An integrated system for digital restoration of prehistoric Theran wall paintings’. Proc. Int. Conf. Digital Signal Processing (DSP), Santorini, Greece, July 2013, pp. 1–6.
    24. 24)
      • 49. Liu, J., Wang, Y., Su, K., et al: ‘Image denoising with multidirectional shrinkage in directionlet domain’, Signal Process., 2016, 125, (C), pp. 64–78.
    25. 25)
      • 3. Muddala, S.-M., Olsson, R., Sjöström, M.: ‘Disocclusion handling using depth-based inpainting’. Proc. Int. Conf. Advances in Multimedia, Venice, Italy, April 2013, pp. 136–141.
    26. 26)
      • 28. Zhang, Y., Pu, Y.-F., Hu, J.-R., et al: ‘A class of fractional-order variational image inpainting models’, Appl. Math. Inf. Sci., 2012, 6, (2), pp. 299–306.
    27. 27)
      • 24. Bai, J., Feng, X.-C.: ‘Fractional-order anisotropic diffusion for image denoising’, IEEE Trans. Image Process., 2007, 16, (10), pp. 2492–2502.
    28. 28)
      • 30. Chen, D., Sun, S.S., Zhang, C.R., et al: ‘Fractional-order TV-L2 model for image denoising’, Cent. Eur. J. Phys., 2013, 11, (10), pp. 1414–1422.
    29. 29)
      • 26. Tian, D., Wu, J.-F., Yang, Y.-J.: ‘A fractional-order Laplacian operator for image edge detection’, Appl. Mech. Mater., 2014, 536–537, pp. 55–58.
    30. 30)
      • 27. Pu, Y.-F., Zhou, J.-L., Yuan, X.: ‘Fractional differential mask: a fractional differential-based approach for multiscale texture enhancement’, IEEE Trans. Image Process., 2010, 19, (2), pp. 491–511.
    31. 31)
      • 12. Boscain, U., Gauthier, J.-P., Prandi, D., et al: ‘Image reconstruction via non-isotropic diffusion in Dubins/Reed–Shepp-like control systems’. Proc. IEEE Conf. Decision and Control (CDC), Los Angeles, California, December 2014, pp. 4278–4283.
    32. 32)
      • 47. He, N., Wang, J.-B., Zhang, L.-L., et al: ‘An improved fractional-order differentiation model for image denoising’, Signal Process., 2015, 112, pp. 180–188.
    33. 33)
      • 44. Snyman, J.A.: ‘Practical mathematical optimization: an introduction to basic optimization theory and classical and new gradient-based algorithms’ (Springer Publishing, 2005), ISBN 0-387-24348-8.
    34. 34)
      • 38. Stein, E.M., Shakarchi, R.: ‘Fourier analysis: an introduction’ (Princeton University Press, Princeton, 2003).
    35. 35)
      • 36. Mathieu, B., Melchior, P., Oustaloup, A., et al: ‘Fractional differentiation for edge detection’, Signal Process., 2003, 83, (11), pp. 2421–2432.
    36. 36)
      • 21. Brox, T., Weickert, J., Burgeth, B., et al: ‘Nonlinear structure tensors’, Image Vis. Comput., 2006, 24, (1), pp. 41–55.
    37. 37)
      • 6. Wang, M., Yan, B., Ngi Ngan, K.: ‘An efficient framework for image/video inpainting’, Signal Process. Image Commun., 2013, 28, (7), pp. 753–762.
    38. 38)
      • 25. Li, H., Yu, Z., Mao, C.: ‘Fractional differential and variational method for image fusion and super-resolution’, Neurocomputing, 2016, 171, (C), pp. 138–148.
    39. 39)
      • 16. Chan, T.-F., Shen, J.: ‘Non-texture inpainting by curvature driven diffusions (CDD)’, J. Vis. Commun. Image Represent., 2001, 12, (4), pp. 436–449.
    40. 40)
      • 20. Weickert, J., Schnörr, C.: ‘A theoretical framework for convex regularizers in PDE-based computation of image motion’, Int. J. Comput. Vis., 2001, 45, (3), pp. 245–264.
    41. 41)
      • 34. Jooyoung, H., Chang-Ock, L.: ‘A nonlinear structure tensor with the diffusivity matrix composed of the image gradient’, J. Math. Imaging Vis., 2009, 34, (2), pp. 137–151.
    42. 42)
      • 29. Chen, D.: ‘Three fractional-order TV-L2 models for image denoising’, J. Comput. Inf. Syst., 2013, 9, (12), pp. 4773–4780.
    43. 43)
      • 15. Mumford, D., Shah, J.: ‘Optimal approximations by piecewise smooth functions and associated variational problems’, Commun. Pure Appl. Math., 1989, 42, (5), pp. 577–685.
    44. 44)
      • 13. Ji, T.-Y., Huang, T.-Z., Zhao, X.-L., et al: ‘Tensor completion using total variation and low-rank matrix factorization’, Inf. Sci., 2016, 326, (C), pp. 243–257.
    45. 45)
      • 5. Wexler, Y., Shechtman, E., Irani, M.: ‘Space-time completion of video’, IEEE Trans. Pattern Anal. Mach. Intell., 2007, 29, (3), pp. 463–476.
    46. 46)
      • 33. Wu, J., Feng, Z., Ren, Z.: ‘Improved structure-adaptive anisotropic filter based on a nonlinear structure tensor’, Cybern. Inf. Technol., 2014, 14, (1), pp. 112–127.
    47. 47)
      • 43. Zhang, X., Burger, M., Osher, S.: ‘A unified primal-dual algorithm framework based on Bregman iteration’, J. Sci. Comput., 2011, 46, (1), pp. 20–46.
    48. 48)
      • 35. Weickert, J.: ‘Anisotropic diffusion in image processing’ (B.G. Teubner Stuttgart, 1998).
    49. 49)
      • 32. Stoll, M., Bosch, J.: ‘A fractional inpainting model based on the vector-valued Cahn-Hilliard equation’, SIAM J. Imaging Sci., 2015, 8, (4), pp. 1–29.
    50. 50)
      • 22. Chierchia, G., Pustelnik, N., Pesquetpopescu, B., et al: ‘A non-local structure tensor based approach for multicomponent image recovery problems’, IEEE Trans. Image Process., 2014, 23, (12), pp. 5531–5544.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2016.1004
Loading

Related content

content/journals/10.1049/iet-ipr.2016.1004
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading