http://iet.metastore.ingenta.com
1887

Improving stereo matching by incorporating geometry prior into ConvNet

Improving stereo matching by incorporating geometry prior into ConvNet

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
Electronics Letters — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Deep learning-based methods for stereo matching have shown superior performance over traditional ones. However, most of them ignore the inherent geometry prior of stereo matching when training, i.e. the reference image can be reconstructed from the second image in the visible regions. The reconstruction can be achieved by backward warping the second image using the disparity map of the reference image, while the visible regions can be calculated by left-right consistency check. This prior is useful especially when the ground truth disparity is sparse (e.g. the outdoor scene such as KITTI 2015). This prior incorporated into a two-stage end-to-end training process, both of which try to minimise the end-point-error with respect to the sparse ground truth disparity (supervised learning), and the reconstruction error (self-supervised learning). The predicted disparity and the reconstruction error of the first stage act as additional information, and are fed to the second stage to make further use of this prior knowledge to improve performance. Experiments on the challenging KITTI 2015 dataset show that the method improves the results in the foreground region, and ranks first among all the published methods on the D1-fg metric.

References

    1. 1)
      • Y. Zhang , W. Qiu , Q. Chen .
        1. Zhang, Y., Qiu, W., Chen, Q., et al: ‘Unrealstereo: a synthetic dataset for analyzing stereo vision’. CoRR abs/1612.04647, 2016.
        .
    2. 2)
      • J. Bontar , Y. Lecun .
        2. Bontar, J., Lecun, Y.: ‘Stereo matching by training a convolutional neural network to compare image patches’, J. Mach. Learn. Res., 2016, 17, (1), pp. 22872318.
        . J. Mach. Learn. Res. , 1 , 2287 - 2318
    3. 3)
      • N. Mayer , E. Ilg , P. Hausser .
        3. Mayer, N., Ilg, E., Hausser, P., et al: ‘A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation’. IEEE Conf. Computer Vision and Pattern Recognition, 2016, pp. 40404048.
        . IEEE Conf. Computer Vision and Pattern Recognition , 4040 - 4048
    4. 4)
      • A. Kendall , H. Martirosyan , S. Dasgupta .
        4. Kendall, A., Martirosyan, H., Dasgupta, S., et al: ‘End-to-end learning of geometry and context for deep stereo regression’. CoRR abs/1703.04309, 2017.
        .
    5. 5)
      • A. Shaked , L. Wolf .
        5. Shaked, A., Wolf, L.: ‘Improved stereo matching with constant highway networks and reflective confidence learning’. IEEE Conf. Computer Vision and Pattern Recognition, 2017.
        . IEEE Conf. Computer Vision and Pattern Recognition
    6. 6)
      • A. Seki , M. Pollefeys .
        6. Seki, A., Pollefeys, M.: ‘Patch based confidence prediction for dense disparity map’. British Machine Vision Conf., 2016, vol. 10.
        . British Machine Vision Conf.
    7. 7)
      • M. Menze , A. Geiger .
        7. Menze, M., Geiger, A.: ‘Object scene flow for autonomous vehicles’. IEEE Conf. Computer Vision and Pattern Recognition, 2015.
        . IEEE Conf. Computer Vision and Pattern Recognition
    8. 8)
    9. 9)
      • R. Garg , G. Carneiro , I. Reid .
        9. Garg, R., Carneiro, G., Reid, I.: ‘Unsupervised CNN for single view depth estimation: geometry to the rescue’. European Conf. Computer Vision, 2016, pp. 740756.
        . European Conf. Computer Vision , 740 - 756
    10. 10)
      • C. Godard , O. Mac Aodha , G.J. Brostow .
        10. Godard, C., Mac Aodha, O., Brostow, G.J.: ‘Unsupervised monocular depth estimation with left–right consistency’. IEEE Conf. Computer Vision and Pattern Recognition, 2017.
        . IEEE Conf. Computer Vision and Pattern Recognition
    11. 11)
      • E. Ilg , N. Mayer , T. Saikia .
        11. Ilg, E., Mayer, N., Saikia, T., et al: ‘Flownet 2.0: evolution of optical flow estimation with deep networks’. IEEE Conf. Computer Vision and Pattern Recognition, 2017.
        . IEEE Conf. Computer Vision and Pattern Recognition
    12. 12)
      • D. Kingma , J. Ba .
        12. Kingma, D., Ba, J.: ‘Adam: a method for stochastic optimization’. 3rd International Conference on Learning Representations (ICLR2015), San Diego, CA, USA, May 2015.
        .
http://iet.metastore.ingenta.com/content/journals/10.1049/el.2017.2418
Loading

Related content

content/journals/10.1049/el.2017.2418
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address