http://iet.metastore.ingenta.com
1887

MSCS: MeshStereo with Cross-Scale Cost Filtering for fast stereo matching

MSCS: MeshStereo with Cross-Scale Cost Filtering for fast stereo matching

For access to this article, please select a purchase option:

Buy eFirst article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
— Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

MeshStereo (MS) and cross-scale cost filtering (CSCF) are two most recently celebrated models for stereo matching. On one hand, MS model enlightens for fast solving the dense stereo correspondence problem according to a region-based opinion. On the other hand, CSCF model could generate more robust matching cost volumes than single scale. In this study, the authors weave these two models together for attaining greater and faster disparity estimation. With CSCF, more powerful initial volumes of matching cost are computed and they are conducted as the data term of MS energy function model. More importantly, the novel-fused stereo model also draws a closer connection between multi-scale aggregated and global algorithms. Integrating the advantages of both stereo models, they name the presented one as MS with cross-scale (MSCS). Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed MSCS outperforms other four most challenging stereo matching algorithms; and also performs better on Microsoft i2i stereo videos. In addition, thanks to this novel-fused model, MSCS requires fewer iteration times for optimising and makes it surprisingly possesses a much faster execution time.

References

    1. 1)
      • D. Scharstein , R. Szeliski .
        1. Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two-frame stereo correspondence algorithms’, Int. J. Comput. Vis. (IJCV), 2002, 47, pp. 742.
        . Int. J. Comput. Vis. (IJCV) , 7 - 42
    2. 2)
      • M. Gong , R. Yang , L. Wang .
        2. Gong, M., Yang, R., Wang, L., et al: ‘A performance study on different cost aggregation approaches used in real-time stereo matching’, Int. J. Comput. Vis. (IJCV), 2007, 75, pp. 283296.
        . Int. J. Comput. Vis. (IJCV) , 283 - 296
    3. 3)
      • F. Tombari , S. Mattoccia , L. Stefano .
        3. Tombari, F., Mattoccia, S., Stefano, L., et al: ‘Classification and evaluation of cost aggregation methods for stereo correspondence’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, USA, 2008, pp. 18.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1 - 8
    4. 4)
      • K.J. Yoon , I.S. Kweon .
        4. Yoon, K.J., Kweon, I.S.: ‘Adaptive support-weight approach for correspondence search’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2006, 28, pp. 650656.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 650 - 656
    5. 5)
      • A. Hosni , M. Bleyer , M. Gelautz .
        5. Hosni, A., Bleyer, M., Gelautz, M., et al: ‘Local stereo matching using geodesic support weights’. IEEE Int. Conf. Image Processing (ICIP), Cairo, Egypt, 2009, pp. 20932096.
        . IEEE Int. Conf. Image Processing (ICIP) , 2093 - 2096
    6. 6)
      • D. Min , J. Lu , M.N. Do .
        6. Min, D., Lu, J., Do, M.N.: ‘A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy?’. IEEE Int. Conf. Computer Vision (ICCV), Barcelona, Spain, 2011, pp. 15671574.
        . IEEE Int. Conf. Computer Vision (ICCV) , 1567 - 1574
    7. 7)
      • D. Min , J. Lu , M.N. Do .
        7. Min, D., Lu, J., Do, M.N.: ‘Joint histogram based cost aggregation for stereo matching’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2013, 35, pp. 25392545.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 2539 - 2545
    8. 8)
      • W. Hu , K. Zhang , L. Sun .
        8. Hu, W., Zhang, K., Sun, L., et al: ‘Virtual support window for adaptive-weight stereo matching’, IEEE Visual Commun. Image Process. (VCIP), Tainan City, Taiwan, 2011, pp. 14.
        . IEEE Visual Commun. Image Process. (VCIP) , 1 - 4
    9. 9)
      • K. Zhang , J. Li , Y. Li .
        9. Zhang, K., Li, J., Li, Y., et al: ‘Binary stereo matching’. IEEE Int. Conf. Pattern Recognition (ICPR), Tsukuba, Japan, 2012, pp. 356359.
        . IEEE Int. Conf. Pattern Recognition (ICPR) , 356 - 359
    10. 10)
      • M. Bleyer , C. Breiteneder .
        10. Bleyer, M., Breiteneder, C.: ‘Stereo matching-state-of-the-art and research challenges’, Chapter 6 of Adv. Top. Comput. Vis., 2013, pp. 143179.
        . Chapter 6 of Adv. Top. Comput. Vis. , 143 - 179
    11. 11)
      • Q. Yang .
        11. Yang, Q.: ‘A non-local cost aggregation method for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 2012, pp. 14021409.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1402 - 1409
    12. 12)
      • X. Mei , X. Sun , W. Dong .
        12. Mei, X., Sun, X., Dong, W., et al: ‘Segment-tree based cost aggregation for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA, 2013, pp. 313320.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 313 - 320
    13. 13)
      • Q. Yang .
        13. Yang, Q.: ‘Local smoothness enforced cost volume regularization for fast stereo correspondence’, IEEE Signal Process. Lett. (SPL), 2015, 22, pp. 14291433.
        . IEEE Signal Process. Lett. (SPL) , 1429 - 1433
    14. 14)
      • P. Yao , H. Zhang , Y. Xue .
        14. Yao, P., Zhang, H., Xue, Y., et al: ‘Iterative color-depth MST cost aggregation for stereo matching’. IEEE Int. Conf. Multimedia & Expo (ICME), Seattle, WA, USA, 2016, pp. 16.
        . IEEE Int. Conf. Multimedia & Expo (ICME) , 1 - 6
    15. 15)
      • P. Yao , H. Zhang , Y. Xue .
        15. Yao, P., Zhang, H., Xue, Y., et al: ‘Segment-tree based cost aggregation for stereo matching with enhanced segmentation advantage’. IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP), New Orleans, LO, USA, 2017, pp. 20272031.
        . IEEE Int. Conf. Acoustics, Speech and Signal Processing (ICASSP) , 2027 - 2031
    16. 16)
      • Q. Yang , L. Wang , R. Yang .
        16. Yang, Q., Wang, L., Yang, R., et al: ‘Real-time global stereo matching using hierarchical belief propagation’. British Machine Vision Conf. (BMVC), Edinburgh, UK, 2006, pp. 110.
        . British Machine Vision Conf. (BMVC) , 1 - 10
    17. 17)
      • Q. Yang , L. Wang , R. Yang .
        17. Yang, Q., Wang, L., Yang, R., et al: ‘Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2008, 31, pp. 492504.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 492 - 504
    18. 18)
      • Q. Yang , L. Wang , N. Ahuja .
        18. Yang, Q., Wang, L., Ahuja, N.: ‘A constant space belief propagation algorithm for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, 2010, pp. 14581465.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1458 - 1465
    19. 19)
      • Y. Boykov , O. Veksler , R. Zabih .
        19. Boykov, Y., Veksler, O., Zabih, R.: ‘Fast approximate energy minimization via graph cuts’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2001, 23, pp. 12221239.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 1222 - 1239
    20. 20)
      • T. Taniai , Y. Matsushita , T. Naemura .
        20. Taniai, T., Matsushita, Y., Naemura, T.: ‘Graph cut based continuous stereo matching using locally shared labels’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 16131620.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1613 - 1620
    21. 21)
      • H. Hirschmuller .
        21. Hirschmuller, H.: ‘Stereo processing by semi-global matching and mutual information’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2008, 30, pp. 328341.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 328 - 341
    22. 22)
      • G. Facciolo , C. Franchis , E. Meinhardt .
        22. Facciolo, G., Franchis, C., Meinhardt, E.: ‘MGM: a significantly more global matching for stereo vision’. British Machine Vision Conf. (BMVC), Swansea, UK, 2015, pp. 112.
        . British Machine Vision Conf. (BMVC) , 1 - 12
    23. 23)
      • Z. Jure , L. Yann .
        23. Jure, Z., Yann, L.: ‘Computing the stereo matching cost with a convolutional neural network’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 15921599.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1592 - 1599
    24. 24)
      • K. Kyung-Rae , K. Chang-Su .
        24. Kyung-Rae, K., Chang-Su, K.: ‘Adaptive smoothness constraints for efficient stereo matching using texture and edge information’. IEEE Int. Conf. Image Processing (ICIP), Phoenix, AZ, USA, 2016, pp. 34293433.
        . IEEE Int. Conf. Image Processing (ICIP) , 3429 - 3433
    25. 25)
      • C. Zhang , Z. Li , Y. Cheng .
        25. Zhang, C., Li, Z., Cheng, Y., et al: ‘MeshStereo: a global stereo model with mesh alignment regularization for view interpolation’. IEEE Int. Conf. Computer Vision (ICCV), Santiago, Chile, 2015, pp. 20572065.
        . IEEE Int. Conf. Computer Vision (ICCV) , 2057 - 2065
    26. 26)
      • K. Zhang , Y. Fang , D. Min .
        26. Zhang, K., Fang, Y., Min, D., et al: ‘Cross-scale cost aggregation for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, 2014, pp. 407414.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 407 - 414
    27. 27)
      • K. Zhang , Y. Fang , D. Min .
        27. Zhang, K., Fang, Y., Min, D., et al: ‘Cross-scale cost aggregation for stereo matching’, IEEE Trans. Circuits Syst. Video Technol. (TCSVT), 2017, 27, pp. 965976.
        . IEEE Trans. Circuits Syst. Video Technol. (TCSVT) , 965 - 976
    28. 28)
      • R. Achanta , A. Shaji , K. Smith .
        28. Achanta, R., Shaji, A., Smith, K., et al: ‘SLIC superpixels compared to state-of-the-art superpixels methods’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2012, 34, pp. 22742282.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 2274 - 2282
    29. 29)
      • M.G. Mozerov , J. Weijer .
        29. Mozerov, M.G., Weijer, J.: ‘Accurate stereo matching by two-step energy minimization’, IEEE Trans. Image Process. (TIP), 2015, 24, pp. 11531163.
        . IEEE Trans. Image Process. (TIP) , 1153 - 1163
    30. 30)
      • F. Besse , C. Rother , A. Fitzgibbon .
        30. Besse, F., Rother, C., Fitzgibbon, A., et al: ‘PMBP: patch match belief propagation for correspondence field estimation’. British Machine Vision Conf. (BMVC), Surrey, UK, 2012, pp. 111.
        . British Machine Vision Conf. (BMVC) , 1 - 11
    31. 31)
      • C. Zhang , Z. Li , R. Cai .
        31. Zhang, C., Li, Z., Cai, R., et al: ‘As-rigid-as-possible stereo under second order smoothness priors’. European Conf. Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 112126.
        . European Conf. Computer Vision (ECCV) , 112 - 126
    32. 32)
      • K. Yamaguchi , T. Hazan , D. McAllester .
        32. Yamaguchi, K., Hazan, T., McAllester, D., et al: ‘Continuous Markov random fields for robust stereo estimation’. European Conf. Computer Vision (ECCV), Florence, Italy, 2012, pp. 4558.
        . European Conf. Computer Vision (ECCV) , 45 - 58
    33. 33)
      • K. Yamaguchi , D. McAllester , R. Urtasun .
        33. Yamaguchi, K., McAllester, D., Urtasun, R.: ‘Efficient joint segmentation, occlusion labeling, stereo and flow estimation’. European Conf. Computer Vision (ECCV), Zurich, Switzerland, 2014, pp. 756771.
        . European Conf. Computer Vision (ECCV) , 756 - 771
    34. 34)
      • W. Hu , K. Zhang , L. Sun .
        34. Hu, W., Zhang, K., Sun, L., et al: ‘Comparisons reducing for local stereo matching using hierarchical structure’. IEEE Int. Conf. Multimedia & Expo (ICME), San Jose, CA, USA, 2013, pp. 16.
        . IEEE Int. Conf. Multimedia & Expo (ICME) , 1 - 6
    35. 35)
      • Y.-H. Jen , E. Dunn , P. Fite-Georgel .
        35. Jen, Y.-H., Dunn, E., Fite-Georgel, P., et al: ‘Adaptive scale selection for hierarchical stereo’. British Machine Vision Conf. (BMVC), Dundee, UK, 2011, pp. 110.
        . British Machine Vision Conf. (BMVC) , 1 - 10
    36. 36)
      • L. Tang , M.K. Garvin , K. Lee .
        36. Tang, L., Garvin, M.K., Lee, K., et al: ‘Robust multiscale stereo matching from fundus images with radiometric differences’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2011, 33, pp. 22452258.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 2245 - 2258
    37. 37)
      • M. Bleyer , C. Rhemann , C. Rother .
        37. Bleyer, M., Rhemann, C., Rother, C.: ‘Patchmatch stereo – stereo matching with slanted support windows’. British Machine Vision Conf. (BMVC), Dundee, UK, 2011, pp. 111.
        . British Machine Vision Conf. (BMVC) , 1 - 11
    38. 38)
      • V. Kolmogorov .
        38. Kolmogorov, V.: ‘Convergent tree-reweighted message passing for energy minimization’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2006, 28, pp. 15681583.
        . IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI) , 1568 - 1583
    39. 39)
      • D. Scharstein , R. Szeliski .
        39. Scharstein, D., Szeliski, R.: ‘Learning conditional random fields for stereo’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Minneapolis, MI, USA, 2007, pp. 18.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1 - 8
    40. 40)
      • H. Hirschmuller , D. Scharstein .
        40. Hirschmuller, H., Scharstein, D.: ‘Evaluation of cost functions for stereo matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Minneapolis, MI, USA, 2007, pp. 18.
        . IEEE Conf. Computer Vision and Pattern Recognition (CVPR) , 1 - 8
    41. 41)
      • D. Scharstein , H. Hirschmuller , Y. Kitajima .
        41. Scharstein, D., Hirschmuller, H., Kitajima, Y., et al: ‘High-resolution stereo datasets with subpixel-accurate ground truth’. German Conf. Pattern Recognition (GCPR), Munster, Germany, 2014, pp. 3142.
        . German Conf. Pattern Recognition (GCPR) , 31 - 42
    42. 42)
      • (2016)
        42. Microsoft i2i Stereo Video. Available at http://research.microsoft.com/enus/projects/i2i/data.aspx, accessed2016.
        .
    43. 43)
      • P. Milanfar .
        43. Milanfar, P.: ‘A tour of modern image filtering: new insights and methods, both practical and theoretical’, IEEE Signal Process. Mag., 2013, 30, pp. 106128.
        . IEEE Signal Process. Mag. , 106 - 128
    44. 44)
      • P. Yao , H. Zhang , Y. Xue .
        44. Yao, P., Zhang, H., Xue, Y., et al: ‘SPMVP: spatial patchmatch stereo with virtual pixel aggregation’. Int. Conf. Neural Information Processing (ICONIP), Guangzhou, China, 2017, Part III, pp. 527542.
        . Int. Conf. Neural Information Processing (ICONIP) , 527 - 542
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2017.0599
Loading

Related content

content/journals/10.1049/iet-cvi.2017.0599
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address