access icon openaccess Integrating wavelet transformation with Markov random field analysis for the depth estimation of light-field images

This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)
Received 13/05/2016, Accepted 13/12/2016, Revised 15/11/2016, Published 20/02/2017

This study addresses the problem of recovering the three-dimensional depth data from the images taken by a light-field camera. Unlike the conventional approach to extract the depth information from the spatial and the angular gradients in the epipolar plane images (EPIs), this study proposes to check the similarity between the pixels for the estimation of EPI slopes and use a wavelet transformation augmented multi-scale analysis to perform smart segmentations. The Markov random field is then applied for surface reconstruction. The proposed algorithm offers significant improvement to the depth estimation, especially for noise contaminated images. Application of the method on the light-field images and on synthesised data show that the proposed method is robust against the noise and achieves better estimation results compared with the available literature.

Inspec keywords: image reconstruction; cameras; Markov processes; wavelet transforms; image segmentation

Other keywords: noise contaminated image; EPI slope; wavelet transformation; Markov random field analysis; spatial gradients; smart segmentation; angular gradients; surface reconstruction; multiscale analysis; light-field camera; three-dimensional depth data recovery; epipolar plane image; light-field image depth estimation; depth information extraction

Subjects: Markov processes; Markov processes; Computer vision and image processing techniques; Integral transforms in numerical analysis; Integral transforms in numerical analysis; Optical, image and video signal processing

References

    1. 1)
      • 2. Adelson, E.H., Wang, J.Y.A.: ‘Single lens stereo with a plenoptic camera’, IEEE Trans. Pattern Anal. Mach. Intell., 1992, 14, (2), pp. 99–106.
    2. 2)
      • 9. Takeda, Y., Hiura, S., Sato, K.: ‘Fusing depth from defocus and stereo with coded apertures’. Proc. of the IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2013.
    3. 3)
      • 10. Tao, M., Hadap, S., Malik, J., et al: ‘Depth from combining defocus and correspondence using light-field cameras’. 2013 14th IEEE Int. Conf. on Computer Vision, ICCV 2013, 2013.
    4. 4)
      • 6. Watanabe, M., Nayar, S.K.: ‘Rational filters for passive depth from defocus’, Int. J. Comput. Vis., 1998, 27, (3), pp. 203–225.
    5. 5)
      • 17. Bouman, C.A., Shapiro, M.: ‘Multiscale random field model for Bayesian image segmentation’, IEEE Trans. Image Process., 1994, 3, (2), pp. 162–177.
    6. 6)
      • 13. Wanner, S., Stephan, M., Bastian, G.: ‘Datasets and benchmarks for densely sampled 4D light fields’, in Brostein, M., Favie, J., Hromann, K. (EDs.): ‘Annual workshop on vision, modeling, and visualization, VMV’ (The Eurographics Association, Switzerland, 2013).
    7. 7)
      • 12. Benedek, C., Shadaydeh, M., Kato, Z., et al: ‘Multilayer Markov random field models for change detection in optical remote sensing images’, ISPRS J. Photogramm. Remote Sens., 2015, 107, pp. 22–37.
    8. 8)
      • 4. Lazaros, N., Sirakoulis, G.C., Gasteratos, A.: ‘Review of stereo vision algorithms: from software to hardware’, Int. J. Optomechatronics, 2008, 2, (4), pp. 435–462.
    9. 9)
      • 16. Baker, H.H., Bolles, R.C.: ‘Generalizing epipolar-plane image-analysis on the spatiotemporal surface’, Int. J. Comput. Vis., 1989, 3, (1), pp. 33–49.
    10. 10)
      • 1. Ng, R.: ‘Digital light field photography’. PhD, Stanford University, 2006.
    11. 11)
      • 14. ‘Lytro’. Available at http://www.lytro.com, accessed 27 November 2013.
    12. 12)
      • 18. Li, S.Z.: ‘Markov random field modeling in image analysis’ (Springer-Verlag London Limited, London, 2009).
    13. 13)
      • 8. Gheta, I., Frese, C., Heizmann, M.: ‘Fusion of combined stereo and focus series for depth estimation’. 36th Jahrestagung der Gesellschaft fur Informatik e.V. (GI): Informatik fur Menschen, INFORMATIK 2006 36th Annual Conf. of the German Informatics Society (GI): Informatics for People, INFORMATIK 2006, Gesellschaft fur Informatik (GI), 2006, 2 October 2006–6 October 2006.
    14. 14)
      • 11. Wanner, S., Goldluecke, B.: ‘Globally consistent depth labeling of 4D light fields’. 2012 IEEE Conf. on Computer Vision and Pattern Recognition, CVPR 2012, 2001 L Street N.W., 2012.
    15. 15)
      • 5. Mroz, F., Breckon, T.P.: ‘An empirical comparison of real-time dense stereo approaches for use in the automotive environment’, EURASIP J. Image Video Process., 2012, 2012, (13), pp. 1–19.
    16. 16)
      • 3. Hahne, C., Aggoun, A., Haxha, S., et al: ‘Light field geometry of a standard plenoptic camera’, Opt. Express, 2014, 22, (22), pp. 26659–26673.
    17. 17)
      • 7. Schechner, Y.Y., Kiryati, N.: ‘Depth from defocus vs. stereo: how different really are they?’, Int. J. Comput. Vis., 2000, 39, (2), pp. 141–162.
    18. 18)
      • 15. Geman, S., Geman, D.: ‘Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images’, IEEE Trans. Pattern Anal. Mach. Intell., 1984, PAMI-6, (6), pp. 721–741.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2016.0151
Loading

Related content

content/journals/10.1049/iet-cvi.2016.0151
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading