RESC-net: reconstruction error as skip connection for stereo matching

RESC-net: reconstruction error as skip connection for stereo matching

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
Electronics Letters — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Recently, the stereo matching task has been dramatically promoted by the deep learning methods. Specifically, the encoder–decoder framework with skip connection achieves outstanding performance over others. The skip connection scheme can bring detailed or in other words, residual information for the final prediction, thus improves the performance, which is successfully applied in many other pixel-wise prediction tasks, such as semantic segmentation, depth estimation and so on. In contrast to other tasks, the authors can explicitly obtain the residual information for stereo matching, which is achieved by back-warping the right image and calculating the reconstruction error. The reconstruction error is successfully used as unsupervised loss, but has not been explored for skip connection. In this Letter, the authors show that the reconstruction error in the feature space is very helpful to bring residual information for the final prediction. They validate the effectiveness of using reconstruction error for skip connection by conducting experiments on the KITTI 2015 and Scene Flow datasets. Experiments show that the proposed scheme can improve the performance by a notable margin and achieves the state-of-the-art performance with very fast processing time.


    1. 1)
    2. 2)
      • 2. Bontar, J., Lecun, Y.: ‘Stereo matching by training a convolutional neural network to compare image patches’, J. Mach. Learn. Res., 2016, 17, (1), 3, pp. 22872318.
    3. 3)
      • 3. Mayer, N., Ilg, E., Hausser, P., et al: ‘A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation’, IEEE Conf. Computer Vision and Pattern Recognition, 2016, 64, pp. 40404048.
    4. 4)
      • 4. Long, J., Shelhamer, E., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’. Proc. of the IEEE Conf. Computer Vision and Pattern Recognition, Boston, MA, USA, June 2015, pp. 34313440.
    5. 5)
      • 5. Pang, J., Sun, W., Ren, J., et al: ‘Cascade residual learning: A two-stage convolutional neural network for stereo matching’. Int. Conf. Computer Vision-Workshop on Geometry Meets Deep Learning, Venice, Italy, October 2017, 64, Vol. 3. No. 9, 2017.
    6. 6)
      • 6. Kendall, A., Martirosyan, H., Dasgupta, S., et al: ‘End-to-end learning of geometry and context for deep stereo regression’. IEEE International Conference on Computer Vision (ICCV), Venica, Italy, October 2017..

Related content

This is a required field
Please enter a valid email address