Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free As-global-as-possible stereo matching with adaptive smoothness prior

  • Author(s): Peng Yao 1, 2 ; Hua Zhang 1, 2, 3 ; Yanbing Xue 1, 2 ; Shengyong Chen 1, 2
    • View affiliations
  • Source: Volume 13, Issue 1, 10 January 2019, p. 98 – 107
    DOI:  10.1049/iet-ipr.2018.5801 , Print ISSN 1751-9659, Online ISSN 1751-9667
© The Institution of Engineering and Technology
Received 03/07/2018, Accepted 08/10/2018, Revised 20/09/2018, Published 12/10/2018

More global matching (MGM) overcomes the limitation of one-dimensional scanline optimisation in semi-global matching (SGM). Nevertheless, the possible weaknesses of the MGM algorithm are as follows: (i) only two directions are considered for each image traversal direction, which may lead to massive mismatches; (ii) disparity estimation around the object boundaries usually performs terrible since the smoothness term is designed independent of the image prior. In this research, the authors consider all of the four directions for each image traversal direction through a novel model. Besides utilising the prior of neighboured pixels' correlation, adaptive smoothness terms are modelled and augmented into the energy function. These contributions encourage ‘as-global-as-possible (AGAP)’. More importantly, different from the recent works in which the aggregated algorithms have been conducted as the data term of an energy function, conversely, the authors make the energy function as a part of cost aggregation framework. Performance evaluations on Middlebury v.2 and v.3 stereo data sets demonstrate that the proposed AGAP outperforms other four most challenging stereo matching algorithms, and also performs better on Microsoft i2i stereo videos. In addition, under various strategies of parallelisation, the presented AGAP shows a near real-time execution time.

Inspec keywords: stereo image processing; image matching

Other keywords: semiglobal matching; i2i stereo videos; object boundaries; performance evaluations; adaptive smoothness; global matching; smoothness term; 3 stereo data sets; data term; image traversal direction; one-dimensional scanline optimisation; massive mismatches; MGM algorithm; aggregated algorithms; disparity estimation; challenging stereo matching algorithms; energy function; possible weaknesses; global-as-possible stereo matching; neighboured pixels

Subjects: Optical, image and video signal processing; Computer vision and image processing techniques

References

    1. 1)
      • 48. Drory, A., Haubold, C., Avidan, S., et al: ‘Semi-global matching: a principled derivation in terms of message passing’. Proc. German Conf. Pattern Recognition, 2014, pp. 43–53.
    2. 2)
      • 16. Mei, X., Sun, X., Dong, W., et al: ‘Segment-tree based cost aggregation for stereo matching’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2013, pp. 313–320.
    3. 3)
      • 8. Hosni, A., Christo, R., Bleyer, M., et al: ‘Fast cost-volume filtering for visual correspondence and beyond’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 32, pp. 504–511.
    4. 4)
      • 25. Yang, Q., Wang, L., Ahuja, N.: ‘A constant space belief propagation algorithm for stereo matching’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010, pp. 1458–1465.
    5. 5)
      • 26. Kolmogorov, V., Zabih, R.: ‘Computing visual correspondence with occlusions using graph cuts’. Proc. IEEE Int. Conf. Computer Vision, 2001, pp. 508–515.
    6. 6)
      • 5. Hosni, A., Bleyer, M., Gelautz, M., et al: ‘Local stereo matching using geodesic support weights’. Proc. IEEE Int. Conf. Image Processing, 2009, pp. 2093–2096.
    7. 7)
      • 10. Min, D., Lu, J., Minh, N.D.: ‘Joint histogram based cost aggregation for stereo matching’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, pp. 2539–2545.
    8. 8)
      • 42. Veksler, O.: ‘Stereo correspondence by dynamic programming on a tree’. Proc. IEEE Conf. Computer Vision and Pattern Recognition), 2005, pp. 384–390.
    9. 9)
      • 3. Tombari, F., Mattoccia, S., Stefano, L., et al: ‘Classification and evaluation of cost aggregation methods for stereo correspondence’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008, pp. 1–8.
    10. 10)
      • 9. Min, D., Lu, J., Minh, N.D.: ‘A revisit to cost aggregation in stereo matching: how far can we reduce its computational redundancy?’ Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 1567–1574.
    11. 11)
      • 11. Helala, M., Qureshi, F.: ‘Accelerating cost volume filtering using salient subvolumes and robust occlusion handling’. Proc. Asian Conf. Computer Vision, 2014, pp. 316–331.
    12. 12)
      • 32. Bleyer, M., Rhemann, C., Rother, C.: ‘Patchmatch stereo – stereo matching with slanted support windows’. Proc. British Machine Vision Conf., 2011, pp. 1–11.
    13. 13)
      • 13. Bleyer, M., Breiteneder, C.: ‘Stereo matching-state-of-the-art and research challenges’, ‘Advanced topics in computer vision’, (Springer, Berlin, 2013), ch. 6, pp. 143–179.
    14. 14)
      • 21. Zhang, K., Fang, Y., Min, D., et al: ‘Cross-scale cost aggregation for stereo matching’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2014, pp. 407–414.
    15. 15)
      • 28. Taniai, T., Matsushita, Y., Naemura, T.: ‘Graph cut based continuous stereo matching using locally shared labels’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2014, pp. 1613–1620.
    16. 16)
      • 20. Yao, P., Zhang, H., Xue, Y., et al: ‘Segment-tree based cost aggregation for stereo matching with enhanced segmentation advantage’. Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, 2017, pp. 2027–2031.
    17. 17)
      • 29. Mozerov, M.G., Weijer, J.: ‘Accurate stereo matching by two-step energy minimization’, IEEE Trans. Image Process., 2015, 24, pp. 1153–1163.
    18. 18)
      • 38. Facciolo1, G., Franchis, C., Meinhardt, E.: ‘MGM: a significantly more global matching for stereo vision’. Proc. British Machine Vision Conf., 2015, pp. 1–12.
    19. 19)
      • 37. LeGendre, C., Batsos, K., Mordohai, P.: ‘High-resolution stereo matching based on sampled photo-consistency computation’. Proc. British Machine Vision Conf., 2017, pp. 1–13.
    20. 20)
      • 1. Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two-frame stereo correspondence algorithms’, Int. J. Comput. Vis., 2002, 47, pp. 7–42.
    21. 21)
      • 14. Yang, Q.: ‘A non-local cost aggregation method for stereo matching’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012, pp. 1402–1409.
    22. 22)
      • 45. Viola, P., Wells, W.M.: ‘Alignment by maximization of mutual information’, Int. J. Comput. Vis., 1997, 24, pp. 137–154.
    23. 23)
      • 24. Yang, Q., Wang, L., Yang, R., et al: ‘Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling’, IEEE Trans. Pattern Anal. Mach. Intell., 2008, 31, pp. 492–504.
    24. 24)
      • 49. Scharstein, D., Szeliski, R.: ‘Learning conditional random fields for stereo’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1–8.
    25. 25)
      • 44. Wang, L., Yang, R., Gong, M., et al: ‘Real-time stereo using approximated joint bilateral filtering and dynamic programming’, J. Real-Time Image Process., 2014, 9, pp. 447–461.
    26. 26)
      • 46. Mozerov, M., Kober, V., Choi, T.S.: ‘Improved motion stereo matching based on a modified dynamic programming’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000, vol. 2, pp. 501–505.
    27. 27)
      • 34. Yao, P., Zhang, H., Xue, Y., et al: ‘MSCS: MeshStereo with cross-scale cost filtering for fast stereo matching’, IET Comput. Vis., 2018, 12, pp. 908–918.
    28. 28)
      • 55. Open Multi-Processing. Available at https://www.openmp.org/, 2018.
    29. 29)
      • 12. Li, X., Liu, J., Chen, G., et al: ‘Efficient methods using slanted support windows for slanted surfaces’, IET Comput. Vis., 2016, 10, pp. 384–391.
    30. 30)
      • 6. He, K., Sun, J., Tang, X.: ‘Guided image filtering’. Proc. European Conf. Computer Vision, 2010, pp. 1–14.
    31. 31)
      • 17. Yang, Q.: ‘local smoothness enforced cost volume regularization for fast stereo correspondence’, IEEE Signal Process. Lett., 2015, 22, pp. 1429–1433.
    32. 32)
      • 53. Zabih, R., Woodfill, J.: ‘Non-parametric local transforms for computing visual correspondence’. Proc. European Conf. Computer Vision, 1994, pp. 151–158.
    33. 33)
      • 54. Intel Threading Building Blocks. Available at https://www.threadingbuild ingblocks.org/, 2018.
    34. 34)
      • 39. Hirschmuller, H.: ‘Accurate and efficient stereo processing by semi-global matching and mutual information’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005, pp. 807–814.
    35. 35)
      • 7. Christo, R., Hosni, A., Bleyer, M., et al: ‘Fast cost-volume filtering for visual correspondence and beyond’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011, pp. 3017–3024.
    36. 36)
      • 15. Yang, Q.: ‘Stereo matching using tree filtering’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 37, pp. 834–846.
    37. 37)
      • 19. Yao, P., Zhang, H., Xue, Y., et al: ‘Iterative color-depth MST cost aggregation for stereo matching’. Proc. IEEE Int. Conf. Multimedia and Expo, 2016, pp. 1–6.
    38. 38)
      • 4. Yoon, K.J., Kweon, I.S.: ‘Adaptive support-weight approach for correspondence search’, IEEE Trans, Pattern Anal. Mach. Intell., 2006, 28, pp. 650–656.
    39. 39)
      • 33. Achanta, R., Shaji, A., Smith, K., et al: ‘SLIC superpixels compared to state-of-the-art superpixels methods’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, pp. 2274–2282.
    40. 40)
      • 30. Yao, P., Zhang, H., Xue, Y., et al: ‘AGO: accelerating global optimization for accurate stereo matching’. Proc. Int. Conf. Multimedia Modeling, 2018, pp. 67–80.
    41. 41)
      • 51. Scharstein, D., Hirschmuller, H., Kitajima, Y., et al: ‘High-resolution stereo datasets with subpixel-accurate ground truth’. Proc. German Conf. Pattern Recognition, 2014, pp. 31–42.
    42. 42)
      • 18. Chen, D., Ardabilian, M., Wang, X., et al: ‘An improved non-local cost aggregation method for stereo matching based on color and boundary cue’. Proc. IEEE Int. Conf. Multimedia and Expo, 2013, pp. 1–6.
    43. 43)
      • 27. Boykov, Y., Veksler, O., Zabih, R.: ‘Fast approximate energy minimization via graph cuts’, IEEE Trans. Pattern Anal. Mach. Intell., 2001, 23, pp. 1222–1239.
    44. 44)
      • 43. Bleyer, M., Gelautz, M.: ‘Simple but effective tree structures for dynamic programming based stereo matching’.Proc. Int. Conf. Computer Vision Theory and Applications, 2008, pp. 415–422.
    45. 45)
      • 2. Gong, M., Yang, R., Wang, L., et al: ‘A performance study on different cost aggregation approaches used in real-time stereo matching’, Int. J. Comput. Vis., 2007, 75, pp. 283–296.
    46. 46)
      • 50. Hirschmuller, H., Scharstein, D.: ‘Evaluation of cost functions for stereo matching’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, pp. 1–8.
    47. 47)
      • 52. Microsoft i2i Stereo Video. Available at http://research.microsoft.com/enus/projects/i2i/data.aspx, 2016.
    48. 48)
      • 22. Zhang, K., Fang, Y., Min, D., et al: ‘Cross-scale cost aggregation for stereo matching’, IEEE Trans. Circuits Syst. Video Technol., 2017, 27, pp. 965–976.
    49. 49)
      • 47. Mozerov, M.: ‘An effective stereo matching algorithm with optimal path cost aggregation’. Proc. Joint Pattern Recognition Symp., 2006, pp. 617–626.
    50. 50)
      • 23. Yang, Q., Wang, L., Yang, R., et al: ‘Real-time global stereo matching using hierarchical belief propagation’. Proc. British Machine Vision Conf., 2006, pp. 1–10.
    51. 51)
      • 36. Kyung-Rae, K., Chang-Su, K.: ‘Adaptive smoothness constraints for efficient stereo matching using texture and edge information’. Proc. IEEE Int. Conf. Image Processing, 2016, pp. 3429–3433.
    52. 52)
      • 31. Zhang, C., Li, Z., Cheng, Y., et al: ‘MeshStereo: a global stereo model with mesh alignment regularization for view interpolation’. Proc. IEEE Int. Conf. Computer Vision, 2015, pp. 2057–2065.
    53. 53)
      • 41. Ohta, Y., Kanade, T.: ‘Stereo by two-level dynamic programming’. Proc. Int. Joint Conf. Artificial Intelligence, 1985, pp. 1120–1126.
    54. 54)
      • 35. Jure, Z., Yann, L.: ‘Computing the stereo matching cost with a convolutional neural network’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 1592–1599.
    55. 55)
      • 40. Hirschmuller, H.: ‘Stereo processing by semi-global matching and mutual information’, IEEE Trans. Pattern Anal. Mach. Intell., 2008, 30, pp. 328–341.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.5801
Loading

Related content

content/journals/10.1049/iet-ipr.2018.5801
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address