Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Depth order estimation for video frames using motion occlusions

© The Institution of Engineering and Technology
Received 30/11/2012, Accepted 20/06/2013, Revised 20/06/2013, Published 29/08/2013

This study proposes a system to estimate the depth order of regions belonging to a monocular image sequence. For each frame, the regions are ordered according to their relative depth using information from the previous and following frames. The algorithm estimates occlusions relying on a hierarchical region-based representation of the image by means of a binary tree. This representation is used to define the final depth order partition which is obtained through an energy minimisation process. Finally, to achieve a global and consistent depth ordering, a depth order graph is constructed and used to eliminate contradictory local cues. The system is evaluated and compared with the state-of-the-art figure/ground labelling systems showing very good results.

Inspec keywords: video signal processing; minimisation; image representation; image motion analysis; trees (mathematics); image sequences

Other keywords: global depth ordering; monocular image sequence; depth order graph; depth order partition; motion occlusions; video frames; consistent depth ordering; binary tree; figure-ground labelling systems; energy minimisation process; hierarchical region-based representation; relative depth

Subjects: Video signal processing; Optimisation techniques; Optical, image and video signal processing; Computer vision and image processing techniques; Combinatorial mathematics; Combinatorial mathematics; Optimisation techniques

References

    1. 1)
    2. 2)
    3. 3)
    4. 4)
    5. 5)
      • 18. Palou, G., Salembier, P.: ‘2.1 depth estimation of frames in image sequences using motion occlusions’, in Fusiello, A., Murino, V., Cucchiara, R. (Eds.): ‘ECCV Workshops’ (Springer, 2012) (LNCS, 7585), pp. 516–525.
    6. 6)
    7. 7)
    8. 8)
      • 28. Maire, M.R.: ‘Contour detection and image segmentation’. PhD thesis, University of California, Berkeley, 2009.
    9. 9)
      • 25. Terruggia, R.: ‘Reliability analysis of probabilistic networks’. PhD thesis, Universita degli Studi di Torino, 2010.
    10. 10)
      • 24. Basha, T., Moses, Y., Avidan, S.: ‘Photo sequencing’, in Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (Eds.): ‘ECCV’ (Springer Berlin, Heidelberg, 2012) (LNCS, 7577), pp. 654–667.
    11. 11)
    12. 12)
    13. 13)
    14. 14)
    15. 15)
    16. 16)
      • 17. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: ‘High accuracy optical flow estimation based on a theory for warping’. European Conf. Computer Vision, Prague, Czech Republic, May 2004, vol. 3024, pp. 25–36.
    17. 17)
      • 19. Vilaplana, V., Marques, F., Salembier, P.: ‘Binary partition trees for object detection’, IEEE Trans. Image Process., 2008, 17, (11), pp. 2201–2216 (doi: 10.1109/TIP.2008.2002841).
    18. 18)
      • 16. Palou, G., Salembier, P.: ‘Monocular depth ordering using T-junctions and convexity occlusion cues’, IEEE Trans. Image Process., 2013, 22, (5), pp. 1926–1939 (doi: 10.1109/TIP.2013.2240002).
    19. 19)
      • 18. Palou, G., Salembier, P.: ‘2.1 depth estimation of frames in image sequences using motion occlusions’, in Fusiello, A., Murino, V., Cucchiara, R. (Eds.): ‘ECCV Workshops’ (Springer, 2012) (LNCS, 7585), pp. 516–525.
    20. 20)
      • 4. Wang, O., Lang, M., Frei, M., Hornung, A., Smolic, A., Gross, M.: ‘StereoBrush: interactive 2D to 3D conversion using discontinuous warps’. Proc. Eighth Eurographics Symp. on Sketch-Based Interfaces and Modeling (SBIM'11), New York, NY, USA, 2011, pp. 47–54.
    21. 21)
      • 22. Andersen, R.: ‘Modern methods for robust regression. Number 152 in quantitative applications in the social sciences’ (Sage Publications, 2008).
    22. 22)
      • 12. Sundberg, P., Brox, T., Maire, M., Arbelaez, P., Malik, J.: ‘Occlusion boundary detection and figure/ground assignment from optical flow’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2011, pp. 2233–2240.
    23. 23)
      • 6. Karsch, K., Liu, C., Kang, S.B.: ‘Depth extraction from video using nonparametric sampling’. ECCV, 2012.
    24. 24)
      • 27. Stein, A.N., Hebert, M.: ‘Occlusion boundaries from motion: low-level detection and mid-level reasoning’, IJCV, 2009, 82, (3), pp. 325–357 (doi: 10.1007/s11263-008-0203-z).
    25. 25)
      • 8. Chang, J.-Y., Cheng, C.-C., Chien, S.-Y., Chen, L.-G.: ‘Relative depth layer extraction for monoscopic video by use of multidimensional filter’. Proc. IEEE Int Multimedia and Expo Conf., 2006, pp. 221–224.
    26. 26)
      • 3. Ward, B., Bing Kang, S., Bennett, E.P.: ‘Depth director: a system for adding depth to movies’, IEEE Comput. Graph. Appl., 2011, 31, (1), pp. 36–48 (doi: 10.1109/MCG.2010.103).
    27. 27)
      • 23. Dwork, C., Kumar, R., Naor, M., Sivakumar, D.: ‘Rank aggregation methods for the web’. Proc. 10th Int. Conf. World Wide Web (WWW'01), New York, NY, USA, 2001, pp. 613–622.
    28. 28)
      • 20. Calderero, F., Marques, F.: ‘Region merging techniques using information theory statistical measures’, IEEE Trans. Image Process., 2010, 19, (6), pp. 1567–1586 (doi: 10.1109/TIP.2010.2043008).
    29. 29)
      • 25. Terruggia, R.: ‘Reliability analysis of probabilistic networks’. PhD thesis, Universita degli Studi di Torino, 2010.
    30. 30)
      • 13. Palou, G., Salembier, P.: ‘Depth ordering on image sequences using motion occlusions’. Proc. 19th IEEE Int. Conf. Image Processing, Florida, USA, September 2012, pp. 1217–1220.
    31. 31)
      • 11. He, X., Yuille, A.: ‘Occlusion boundary detection using pseudo-depth’. ECCV, 2010 (LNCS, 6314), pp. 539–552.
    32. 32)
      • 28. Maire, M.R.: ‘Contour detection and image segmentation’. PhD thesis, University of California, Berkeley, 2009.
    33. 33)
      • 1. Ono, M.E., Rivest, J., Ono, H.: ‘Depth perception as a function of motion parallax and absolute-distance information’, J. Exp. Psychol., Hum. Percept. Perform., 1986, 12, pp. 331–337 (doi: 10.1037/0096-1523.12.3.331).
    34. 34)
      • 5. Bergen, L., Meyer, F.: ‘A novel approach to depth ordering in monocular image sequences’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2000, vol. 2, pp. 536–541.
    35. 35)
      • 24. Basha, T., Moses, Y., Avidan, S.: ‘Photo sequencing’, in Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (Eds.): ‘ECCV’ (Springer Berlin, Heidelberg, 2012) (LNCS, 7577), pp. 654–667.
    36. 36)
      • 10. Zhang, G., Jia, J., Wong, T.-T., Bao, H.: ‘Consistent depth maps recovery from a video sequence’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (6), pp. 974–988 (doi: 10.1109/TPAMI.2009.52).
    37. 37)
      • 2. Qian, N., Qian, D.N.: ‘Binocular disparity and the perception of depth’, Neuron, 1997, 18, pp. 359–368 (doi: 10.1016/S0896-6273(00)81238-6).
    38. 38)
      • 21. Kanatani, K.: ‘Transformation of optical flow by camera rotation’, IEEE Trans. Pattern Anal. Mach. Intell., 1988, 10, (2), pp. 131–143 (doi: 10.1109/34.3879).
    39. 39)
      • 26. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: ‘Introduction to algorithms’ (MIT Press, 2001, 2nd edn.).
    40. 40)
      • 7. Turetken, E., Alatan, A.A.: ‘Temporally consistent layer depth ordering via pixel voting for pseudo 3D representation’. 3DTV Conf., 2009, pp. 1–4.
    41. 41)
      • 14. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (5), pp. 898–916 (doi: 10.1109/TPAMI.2010.161).
    42. 42)
      • 15. Salembier, P., Garrido, L.: ‘Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval’, IEEE Trans. Image Process., 2000, 9, (4), pp. 561–576 (doi: 10.1109/83.841934).
    43. 43)
      • 9. Li, P., Farin, D., Gunnewiek, R.K., de With, P.H.N.: ‘On creating depth maps from monoscopic video using structure from motion’. Proc. 27th Symp. on Information Theory in the Benelux, 2006, pp. 508–515.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2012.0287
Loading

Related content

content/journals/10.1049/iet-cvi.2012.0287
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address