access icon free MSCS: MeshStereo with Cross-Scale Cost Filtering for fast stereo matching

MeshStereo (MS) and cross-scale cost filtering (CSCF) are two most recently celebrated models for stereo matching. On one hand, MS model enlightens for fast solving the dense stereo correspondence problem according to a region-based opinion. On the other hand, CSCF model could generate more robust matching cost volumes than single scale. In this study, the authors weave these two models together for attaining greater and faster disparity estimation. With CSCF, more powerful initial volumes of matching cost are computed and they are conducted as the data term of MS energy function model. More importantly, the novel-fused stereo model also draws a closer connection between multi-scale aggregated and global algorithms. Integrating the advantages of both stereo models, they name the presented one as MS with cross-scale (MSCS). Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed MSCS outperforms other four most challenging stereo matching algorithms; and also performs better on Microsoft i2i stereo videos. In addition, thanks to this novel-fused model, MSCS requires fewer iteration times for optimising and makes it surprisingly possesses a much faster execution time.

Inspec keywords: stereo image processing; image matching; video signal processing; image filtering

Other keywords: MS energy function model; disparity estimation; Middlebury v.3 stereo data sets; Microsoft i2i stereo videos; MS with cross-scale cost filtering; multiscale aggregated algorithm; matching cost; fast stereo matching; novel-fused stereo model; MeshStereo; robust matching cost volumes; data term; Middlebury v.2 stereo data sets; region-based opinion; MSCS; CSCF model; global algorithms; dense stereo correspondence problem

Subjects: Image recognition; Computer vision and image processing techniques; Filtering methods in signal processing; Video signal processing

References

    1. 1)
      • 20. Taniai, T., Matsushita, Y., Naemura, T.: ‘Graph cut based continuous stereo matching using locally shared labels’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 16131620.
    2. 2)
      • 22. Facciolo, G., Franchis, C., Meinhardt, E.: ‘MGM: a significantly more global matching for stereo vision’. British Machine Vision Conf. (BMVC), Swansea, UK, 2015, pp. 112.
    3. 3)
      • 18. Yang, Q., Wang, L., Ahuja, N.: ‘A constant space belief propagation algorithm for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, 2010, pp. 14581465.
    4. 4)
      • 15. Yao, P., Zhang, H., Xue, Y., et al: ‘Segment-tree based cost aggregation for stereo matching with enhanced segmentation advantage’. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LO, USA, 2017, pp. 20272031.
    5. 5)
      • 21. Hirschmuller, H.: ‘Stereo processing by semi-global matching and mutual information’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2008, 30, pp. 328341.
    6. 6)
      • 2. Gong, M., Yang, R., Wang, L., et al: ‘A performance study on different cost aggregation approaches used in real-time stereo matching’, Int. J. Comput. Vis. (IJCV), 2007, 75, pp. 283296.
    7. 7)
      • 8. Hu, W., Zhang, K., Sun, L., et al: ‘Virtual support window for adaptive-weight stereo matching’, IEEE Visual Commun. Image Process. (VCIP), Tainan City, Taiwan, 2011, pp. 14.
    8. 8)
      • 12. Mei, X., Sun, X., Dong, W., et al: ‘Segment-tree based cost aggregation for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, 2013, pp. 313320.
    9. 9)
      • 13. Yang, Q.: ‘Local smoothness enforced cost volume regularization for fast stereo correspondence’, IEEE Signal Process. Lett. (SPL), 2015, 22, pp. 14291433.
    10. 10)
      • 38. Kolmogorov, V.: ‘Convergent tree-reweighted message passing for energy minimization’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2006, 28, pp. 15681583.
    11. 11)
      • 41. Scharstein, D., Hirschmuller, H., Kitajima, Y., et al: ‘High-resolution stereo datasets with subpixel-accurate ground truth’. German Conf. Pattern Recognition (GCPR), Munster, Germany, 2014, pp. 3142.
    12. 12)
      • 10. Bleyer, M., Breiteneder, C.: ‘Stereo matching-state-of-the-art and research challenges’, Chapter 6 of Adv. Top. Comput. Vis., 2013, pp. 143179.
    13. 13)
      • 1. Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two-frame stereo correspondence algorithms’, Int. J. Comput. Vis. (IJCV), 2002, 47, pp. 742.
    14. 14)
      • 6. Min, D., Lu, J., Do, M.N.: ‘A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy?’. IEEE Int. Conf. Computer Vision (ICCV), Barcelona, Spain, 2011, pp. 15671574.
    15. 15)
      • 29. Mozerov, M.G., Weijer, J.: ‘Accurate stereo matching by two-step energy minimization’, IEEE Trans. Image Process. (TIP), 2015, 24, pp. 11531163.
    16. 16)
      • 44. Yao, P., Zhang, H., Xue, Y., et al: ‘SPMVP: spatial patchmatch stereo with virtual pixel aggregation’. Int. Conf. Neural Information Processing (ICONIP), Guangzhou, China, 2017, Part III, pp. 527542.
    17. 17)
      • 17. Yang, Q., Wang, L., Yang, R., et al: ‘Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2008, 31, pp. 492504.
    18. 18)
      • 24. Kyung-Rae, K., Chang-Su, K.: ‘Adaptive smoothness constraints for efficient stereo matching using texture and edge information’. IEEE Int. Conf. Image Processing (ICIP), Phoenix, AZ, USA, 2016, pp. 34293433.
    19. 19)
      • 7. Min, D., Lu, J., Do, M.N.: ‘Joint histogram based cost aggregation for stereo matching’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2013, 35, pp. 25392545.
    20. 20)
      • 42. Microsoft i2i Stereo Video. Available at http://research.microsoft.com/enus/projects/i2i/data.aspx, accessed2016.
    21. 21)
      • 25. Zhang, C., Li, Z., Cheng, Y., et al: ‘MeshStereo: a global stereo model with mesh alignment regularization for view interpolation’. IEEE Int. Conf. Computer Vision (ICCV), Santiago, Chile, 2015, pp. 20572065.
    22. 22)
      • 27. Zhang, K., Fang, Y., Min, D., et al: ‘Cross-scale cost aggregation for stereo matching’, IEEE Trans. Circuits Syst. Video Technol. (TCSVT), 2017, 27, pp. 965976.
    23. 23)
      • 30. Besse, F., Rother, C., Fitzgibbon, A., et al: ‘PMBP: patch match belief propagation for correspondence field estimation’. British Machine Vision Conf. (BMVC), Surrey, UK, 2012, pp. 111.
    24. 24)
      • 16. Yang, Q., Wang, L., Yang, R., et al: ‘Real-time global stereo matching using hierarchical belief propagation’. British Machine Vision Conf. (BMVC), Edinburgh, UK, 2006, pp. 110.
    25. 25)
      • 40. Hirschmuller, H., Scharstein, D.: ‘Evaluation of cost functions for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Minneapolis, MI, USA, 2007, pp. 18.
    26. 26)
      • 19. Boykov, Y., Veksler, O., Zabih, R.: ‘Fast approximate energy minimization via graph cuts’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2001, 23, pp. 12221239.
    27. 27)
      • 3. Tombari, F., Mattoccia, S., Stefano, L., et al: ‘Classification and evaluation of cost aggregation methods for stereo correspondence’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, USA, 2008, pp. 18.
    28. 28)
      • 32. Yamaguchi, K., Hazan, T., McAllester, D., et al: ‘Continuous Markov random fields for robust stereo estimation’. European Conf. Computer Vision (ECCV), Florence, Italy, 2012, pp. 4558.
    29. 29)
      • 23. Jure, Z., Yann, L.: ‘Computing the stereo matching cost with a convolutional neural network’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 15921599.
    30. 30)
      • 9. Zhang, K., Li, J., Li, Y., et al: ‘Binary stereo matching’. IEEE Int. Conf. Pattern Recognition (ICPR), Tsukuba, Japan, 2012, pp. 356359.
    31. 31)
      • 43. Milanfar, P.: ‘A tour of modern image filtering: new insights and methods, both practical and theoretical’, IEEE Signal Process. Mag., 2013, 30, pp. 106128.
    32. 32)
      • 37. Bleyer, M., Rhemann, C., Rother, C.: ‘Patchmatch stereo – stereo matching with slanted support windows’. British Machine Vision Conf. (BMVC), Dundee, UK, 2011, pp. 111.
    33. 33)
      • 14. Yao, P., Zhang, H., Xue, Y., et al: ‘Iterative color-depth MST cost aggregation for stereo matching’. IEEE Int. Conf. Multimedia & Expo (ICME), Seattle, WA, USA, 2016, pp. 16.
    34. 34)
      • 36. Tang, L., Garvin, M.K., Lee, K., et al: ‘Robust multiscale stereo matching from fundus images with radiometric differences’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2011, 33, pp. 22452258.
    35. 35)
      • 35. Jen, Y.-H., Dunn, E., Fite-Georgel, P., et al: ‘Adaptive scale selection for hierarchical stereo’. British Machine Vision Conf. (BMVC), Dundee, UK, 2011, pp. 110.
    36. 36)
      • 39. Scharstein, D., Szeliski, R.: ‘Learning conditional random fields for stereo’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Minneapolis, MI, USA, 2007, pp. 18.
    37. 37)
      • 34. Hu, W., Zhang, K., Sun, L., et al: ‘Comparisons reducing for local stereo matching using hierarchical structure’. IEEE Int. Conf. Multimedia & Expo (ICME), San Jose, CA, USA, 2013, pp. 16.
    38. 38)
      • 28. Achanta, R., Shaji, A., Smith, K., et al: ‘SLIC superpixels compared to state-of-the-art superpixels methods’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2012, 34, pp. 22742282.
    39. 39)
      • 5. Hosni, A., Bleyer, M., Gelautz, M., et al: ‘Local stereo matching using geodesic support weights’. IEEE Int. Conf. Image Processing (ICIP), Cairo, Egypt, 2009, pp. 20932096.
    40. 40)
      • 4. Yoon, K.J., Kweon, I.S.: ‘Adaptive support-weight approach for correspondence search’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2006, 28, pp. 650656.
    41. 41)
      • 11. Yang, Q.: ‘A non-local cost aggregation method for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 2012, pp. 14021409.
    42. 42)
      • 33. Yamaguchi, K., McAllester, D., Urtasun, R.: ‘Efficient joint segmentation, occlusion labeling, stereo and flow estimation’. European Conf. Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 756771.
    43. 43)
      • 26. Zhang, K., Fang, Y., Min, D., et al: ‘Cross-scale cost aggregation for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 407414.
    44. 44)
      • 31. Zhang, C., Li, Z., Cai, R., et al: ‘As-rigid-as-possible stereo under second order smoothness priors’. European Conf. Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 112126.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2017.0599
Loading

Related content

content/journals/10.1049/iet-cvi.2017.0599
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading