© The Institution of Engineering and Technology
This study proposes a system to estimate the depth order of regions belonging to a monocular image sequence. For each frame, the regions are ordered according to their relative depth using information from the previous and following frames. The algorithm estimates occlusions relying on a hierarchical region-based representation of the image by means of a binary tree. This representation is used to define the final depth order partition which is obtained through an energy minimisation process. Finally, to achieve a global and consistent depth ordering, a depth order graph is constructed and used to eliminate contradictory local cues. The system is evaluated and compared with the state-of-the-art figure/ground labelling systems showing very good results.
References
-
-
1)
-
27. Stein, A.N., Hebert, M.: ‘Occlusion boundaries from motion: low-level detection and mid-level reasoning’, IJCV, 2009, 82, (3), pp. 325–357 (doi: 10.1007/s11263-008-0203-z).
-
2)
-
1. Ono, M.E., Rivest, J., Ono, H.: ‘Depth perception as a function of motion parallax and absolute-distance information’, J. Exp. Psychol., Hum. Percept. Perform., 1986, 12, pp. 331–337 (doi: 10.1037/0096-1523.12.3.331).
-
3)
-
2. Qian, N., Qian, D.N.: ‘Binocular disparity and the perception of depth’, Neuron, 1997, 18, pp. 359–368 (doi: 10.1016/S0896-6273(00)81238-6).
-
4)
-
19. Vilaplana, V., Marques, F., Salembier, P.: ‘Binary partition trees for object detection’, IEEE Trans. Image Process., 2008, 17, (11), pp. 2201–2216 (doi: 10.1109/TIP.2008.2002841).
-
5)
-
18. Palou, G., Salembier, P.: ‘2.1 depth estimation of frames in image sequences using motion occlusions’, in Fusiello, A., Murino, V., Cucchiara, R. (Eds.): ‘ECCV Workshops’ (Springer, 2012) (, 7585), pp. 516–525.
-
6)
-
20. Calderero, F., Marques, F.: ‘Region merging techniques using information theory statistical measures’, IEEE Trans. Image Process., 2010, 19, (6), pp. 1567–1586 (doi: 10.1109/TIP.2010.2043008).
-
7)
-
G. Zhang ,
J. Jia ,
T. Wong ,
H. Bao
.
Consistent depth maps recovery from a video sequence.
IEEE Trans. Pattern Anal. Mach. Intell.
,
6 ,
974 -
988
-
8)
-
28. Maire, M.R.: ‘Contour detection and image segmentation’. , University of California, Berkeley, 2009.
-
9)
-
25. Terruggia, R.: ‘Reliability analysis of probabilistic networks’. , Universita degli Studi di Torino, 2010.
-
10)
-
24. Basha, T., Moses, Y., Avidan, S.: ‘Photo sequencing’, in Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (Eds.): ‘ECCV’ (Springer Berlin, Heidelberg, 2012) (, 7577), pp. 654–667.
-
11)
-
P. Salembier ,
L. Garrido
.
Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval.
IEEE Trans. Image Process.
,
4 ,
561 -
576
-
12)
-
16. Palou, G., Salembier, P.: ‘Monocular depth ordering using T-junctions and convexity occlusion cues’, IEEE Trans. Image Process., 2013, 22, (5), pp. 1926–1939 (doi: 10.1109/TIP.2013.2240002).
-
13)
-
3. Ward, B., Bing Kang, S., Bennett, E.P.: ‘Depth director: a system for adding depth to movies’, IEEE Comput. Graph. Appl., 2011, 31, (1), pp. 36–48 (doi: 10.1109/MCG.2010.103).
-
14)
-
35. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, pp. 898–916 (doi: 10.1109/TPAMI.2010.161).
-
15)
-
21. Kanatani, K.: ‘Transformation of optical flow by camera rotation’, IEEE Trans. Pattern Anal. Mach. Intell., 1988, 10, (2), pp. 131–143 (doi: 10.1109/34.3879).
-
16)
-
17. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: ‘High accuracy optical flow estimation based on a theory for warping’. European Conf. Computer Vision, Prague, Czech Republic, May 2004, vol. 3024, pp. 25–36.
-
17)
-
19. Vilaplana, V., Marques, F., Salembier, P.: ‘Binary partition trees for object detection’, IEEE Trans. Image Process., 2008, 17, (11), pp. 2201–2216 (doi: 10.1109/TIP.2008.2002841).
-
18)
-
16. Palou, G., Salembier, P.: ‘Monocular depth ordering using T-junctions and convexity occlusion cues’, IEEE Trans. Image Process., 2013, 22, (5), pp. 1926–1939 (doi: 10.1109/TIP.2013.2240002).
-
19)
-
18. Palou, G., Salembier, P.: ‘2.1 depth estimation of frames in image sequences using motion occlusions’, in Fusiello, A., Murino, V., Cucchiara, R. (Eds.): ‘ECCV Workshops’ (Springer, 2012) (, 7585), pp. 516–525.
-
20)
-
4. Wang, O., Lang, M., Frei, M., Hornung, A., Smolic, A., Gross, M.: ‘StereoBrush: interactive 2D to 3D conversion using discontinuous warps’. Proc. Eighth Eurographics Symp. on Sketch-Based Interfaces and Modeling (SBIM'11), New York, NY, USA, 2011, pp. 47–54.
-
21)
-
22. Andersen, R.: ‘Modern methods for robust regression. Number 152 in quantitative applications in the social sciences’ (Sage Publications, 2008).
-
22)
-
12. Sundberg, P., Brox, T., Maire, M., Arbelaez, P., Malik, J.: ‘Occlusion boundary detection and figure/ground assignment from optical flow’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2011, pp. 2233–2240.
-
23)
-
6. Karsch, K., Liu, C., Kang, S.B.: ‘Depth extraction from video using nonparametric sampling’. ECCV, 2012.
-
24)
-
27. Stein, A.N., Hebert, M.: ‘Occlusion boundaries from motion: low-level detection and mid-level reasoning’, IJCV, 2009, 82, (3), pp. 325–357 (doi: 10.1007/s11263-008-0203-z).
-
25)
-
8. Chang, J.-Y., Cheng, C.-C., Chien, S.-Y., Chen, L.-G.: ‘Relative depth layer extraction for monoscopic video by use of multidimensional filter’. Proc. IEEE Int Multimedia and Expo Conf., 2006, pp. 221–224.
-
26)
-
3. Ward, B., Bing Kang, S., Bennett, E.P.: ‘Depth director: a system for adding depth to movies’, IEEE Comput. Graph. Appl., 2011, 31, (1), pp. 36–48 (doi: 10.1109/MCG.2010.103).
-
27)
-
23. Dwork, C., Kumar, R., Naor, M., Sivakumar, D.: ‘Rank aggregation methods for the web’. Proc. 10th Int. Conf. World Wide Web (WWW'01), New York, NY, USA, 2001, pp. 613–622.
-
28)
-
20. Calderero, F., Marques, F.: ‘Region merging techniques using information theory statistical measures’, IEEE Trans. Image Process., 2010, 19, (6), pp. 1567–1586 (doi: 10.1109/TIP.2010.2043008).
-
29)
-
25. Terruggia, R.: ‘Reliability analysis of probabilistic networks’. , Universita degli Studi di Torino, 2010.
-
30)
-
13. Palou, G., Salembier, P.: ‘Depth ordering on image sequences using motion occlusions’. Proc. 19th IEEE Int. Conf. Image Processing, Florida, USA, September 2012, pp. 1217–1220.
-
31)
-
11. He, X., Yuille, A.: ‘Occlusion boundary detection using pseudo-depth’. ECCV, 2010 (, 6314), pp. 539–552.
-
32)
-
28. Maire, M.R.: ‘Contour detection and image segmentation’. , University of California, Berkeley, 2009.
-
33)
-
1. Ono, M.E., Rivest, J., Ono, H.: ‘Depth perception as a function of motion parallax and absolute-distance information’, J. Exp. Psychol., Hum. Percept. Perform., 1986, 12, pp. 331–337 (doi: 10.1037/0096-1523.12.3.331).
-
34)
-
5. Bergen, L., Meyer, F.: ‘A novel approach to depth ordering in monocular image sequences’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2000, vol. 2, pp. 536–541.
-
35)
-
24. Basha, T., Moses, Y., Avidan, S.: ‘Photo sequencing’, in Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (Eds.): ‘ECCV’ (Springer Berlin, Heidelberg, 2012) (, 7577), pp. 654–667.
-
36)
-
10. Zhang, G., Jia, J., Wong, T.-T., Bao, H.: ‘Consistent depth maps recovery from a video sequence’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (6), pp. 974–988 (doi: 10.1109/TPAMI.2009.52).
-
37)
-
2. Qian, N., Qian, D.N.: ‘Binocular disparity and the perception of depth’, Neuron, 1997, 18, pp. 359–368 (doi: 10.1016/S0896-6273(00)81238-6).
-
38)
-
21. Kanatani, K.: ‘Transformation of optical flow by camera rotation’, IEEE Trans. Pattern Anal. Mach. Intell., 1988, 10, (2), pp. 131–143 (doi: 10.1109/34.3879).
-
39)
-
26. Cormen, T.H., Leiserson, C.E., Rivest, R.L., Stein, C.: ‘Introduction to algorithms’ (MIT Press, 2001, 2nd edn.).
-
40)
-
7. Turetken, E., Alatan, A.A.: ‘Temporally consistent layer depth ordering via pixel voting for pseudo 3D representation’. 3DTV Conf., 2009, pp. 1–4.
-
41)
-
14. Arbeláez, P., Maire, M., Fowlkes, C., Malik, J.: ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (5), pp. 898–916 (doi: 10.1109/TPAMI.2010.161).
-
42)
-
15. Salembier, P., Garrido, L.: ‘Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval’, IEEE Trans. Image Process., 2000, 9, (4), pp. 561–576 (doi: 10.1109/83.841934).
-
43)
-
9. Li, P., Farin, D., Gunnewiek, R.K., de With, P.H.N.: ‘On creating depth maps from monoscopic video using structure from motion’. Proc. 27th Symp. on Information Theory in the Benelux, 2006, pp. 508–515.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2012.0287
Related content
content/journals/10.1049/iet-cvi.2012.0287
pub_keyword,iet_inspecKeyword,pub_concept
6
6