http://iet.metastore.ingenta.com
1887

Combining 2D and 3D features to improve road detection based on stereo cameras

Combining 2D and 3D features to improve road detection based on stereo cameras

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Computer Vision — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Road detection is a fundamental component of autonomous driving systems since it provides valid space and candidate regions of objects for driving decision. The core of road detection methods is extracting effective and discriminative features. Since two-dimensional (2D) and 3D features are complementary, the authors propose a robust multi-feature combination and optimisation framework for stereo image pairs, called Feature++. First, several 2D and 3D features such as Gabor and plane are, respectively, extracted after the generation of 2D super-pixel and a 3D depth image from stereo matching. Second, the combined features are fed into a three-layer shallow neural network classifier to decide whether a super-pixel is road region or not. Finally, the classified results are further refined using fully connected conditional random field (CRF), taking the content information into consideration. We extensively evaluate the performance of four 2D features, four 3D features, and their combinations. Experiments conducted on the KITTI ROAD benchmark show that (i) the combinations of 2D and 3D features greatly improve the road detection performance and (ii) using CRF as a refinement step is necessary. Overall, their proposed ‘Feature + +’ method outperforms most manually designed features, and is comparable with state-of-the-art methods that are based on deep learning methods.

References

    1. 1)
      • 1. Gao, J., Wang, Q., Yuan, Y.: ‘Embedding structured contour and location prior in siamesed fully convolutional networks for road detection’. Proc. IEEE Int. Conf. Robotics and Automation (ICRA), Singapore, May 2017, pp. 17.
    2. 2)
      • 2. Kocamaz, M., Navarro-Serment, L., Hebert, M.: ‘Map-supervised road detection’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Gothenburg, Sweden, June 2016, pp. 16.
    3. 3)
      • 3. Oliveira, G., Burgard, W., Brox, T.: ‘Efficient deep methods for monocular road segmentation’. Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), Daejeon, South Korea, October 2016, pp. 18.
    4. 4)
      • 4. Muñoz-Bulnes, J., Fernandez, C., Parra, I., et al: ‘Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection’. Proc. IEEE Int. Conf. Intelligent Transportation Systems Workshop on Deep Learning for Autonomous Driving, Yokohama, Japan, October 2017, pp. 16.
    5. 5)
      • 5. Chen, X., Kundu, K., Zhu, Y., et al: ‘3D object proposals for accurate object class detection’. Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS), Montréal, Canada, December 2015, pp. 424432.
    6. 6)
      • 6. Gabor, D.: ‘Theory of communication – part 1: the analysis of information’, J. Inst. Electr. Eng. III, 2010, 93, (26), pp. 429441.
    7. 7)
      • 7. Bo, L., Ren, X., Fox, D.: ‘Kernel descriptors for visual recognition’. Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS), Vancouver, Canada, December 2010, pp. 244252.
    8. 8)
      • 8. He, W.L., Cai, G.R., Zhong, Z., et al: ‘Feature++: cross dimension feature fusion for road detection’. Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), New Orleans, USA, March 2017, pp. 16.
    9. 9)
      • 9. Alvarez, J., Lopez, A.M.: ‘Road detection based on illuminant invariance’, IEEE Trans. Intell. Transp. Syst., 2011, 12, (1), pp. 184193.
    10. 10)
      • 10. Wang, B., Frémont, V., Rodriguez, S.: ‘Color-based road detection and its evaluation on the KITTI road benchmark’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Dearborn, MI, June 2014, pp. 3136.
    11. 11)
      • 11. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Learning appearance models for road detection’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Gold Coast, QLD, Australia, June 2013, pp. 423429.
    12. 12)
      • 12. Alvarez, J.M., Gevers, T., LeCun, Y., et al: ‘Road scene segmentation from a single image’. Proc. European Conf. Computer Vision (ECCV), Firenze, Italy, October 2012, pp. 376389.
    13. 13)
      • 13. Rasmussen, C.: ‘Grouping dominant orientations for ill-structured road following’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Washington DC, USA, June 2004, pp. 18.
    14. 14)
      • 14. Kong, H., Audibert, J.-Y., Ponce, J.: ‘Vanishing point detection for road detection’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Miami, FL, June 2009, pp. 96103.
    15. 15)
      • 15. Fritsch, J., Kuehnl, T., Kummert, F.: ‘Monocular road terrain detection by combining visual and spatial information’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (4), pp. 15861596.
    16. 16)
      • 16. Mohan, R.: ‘Deep deconvolutional networks for scene parsing’. arXiv: Machine Learning, November 2014. Available at https://arxiv.org/pdf/1411.4101.pdf, accessed November 2014.
    17. 17)
      • 17. Dahlkamp, H., Kaehler, A., Stavens, D., et al: ‘Self-supervised monocular road detection in desert terrain’. Proc. IEEE Int. Conf. Robotics Science Systems, Pennsylvania, USA, August 2006, pp. 17.
    18. 18)
      • 18. Alon, Y., Ferencz, A., Shashua, A.: ‘Off-road path following using region classification and geometric projection constraints’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, June 2006, pp. 689696.
    19. 19)
      • 19. Xiao, L., Dai, B., Liu, D., et al: ‘CRF based road detection with multi-sensor fusion’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp. (IV), Seoul, South Korea, June 2015, pp. 192198.
    20. 20)
      • 20. Vitor, G., Lima, D., Victorino, A., et al: ‘A 2D/3D vision based approach applied to road detection in urban environments’, IEEE. Intell. Veh. Symp. (IV), 2013, 36, (1), pp. 952957.
    21. 21)
      • 21. Thrun, S., Montemerlo, M., Dahlkamp, H., et al: ‘Stanley: the robot that won the DARPA grand challenge’ (Stanford University, Stanford, CA, 2007), pp. 143.
    22. 22)
      • 22. Moosmann, F., Pink, O., Stiller, C.: ‘Segmentation of 3d LiDAR data in non-flat urban environments using a local convexity criterion’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Xi'an, China, June 2009, pp. 215220.
    23. 23)
      • 23. Chen, T., Dai, B., Wang, R., et al: ‘Gaussian-process-based real time ground segmentation for autonomous land vehicles’, J. Intell. Robot. Syst. (JINT), 2014, 76, (3), pp. 563582.
    24. 24)
      • 24. Labayrade, R., Aubert, D., Tarel, J.-P.: ‘Real time obstacle detection in stereovision on non-flat road geometry through ‘v-disparity’ representation’. Proc. IEEE Int. Conf. Intelligent Vehicle Symp., Versailles, France, June 2002, pp. 646651.
    25. 25)
      • 25. Badino, H., Franke, U., Mester, R.: ‘Free space computation using stochastic occupancy grids and dynamic programming’. Proc. IEEE Int. Conf. Computer Vision (ICCV) Workshop on Dynamical Vision, Rio de Janeiro, Brazil, October 2007, pp. 18.
    26. 26)
      • 26. Oniga, F., Nedevschi, S.: ‘Processing dense stereo data using elevation maps: road surface, traffic isle, and obstacle detection’, IEEE Trans. Veh. Technol, 2010, 59, (3), pp. 11721182.
    27. 27)
      • 27. Suhr, J.K., Jung, H.G.: ‘Dense stereo-based robust vertical road profile estimation using Hough transform and dynamic programming’, IEEE Trans. Intell. Transp. Syst., 2015, 16, (3), pp. 15281536.
    28. 28)
      • 28. Ramakrishnan, A.G., Kumar, R.S., Raghu, H.V.: ‘Neural network-based segmentation of textures using Gabor features’. Proc. IEEE Int. Workshop on Neural Networks for Signal Processing, Martigny, Switzerland, September 2002, pp. 365374.
    29. 29)
      • 29. Yamaguchi, K., McAllester, D., Urtasun, R.: ‘Efficient joint segmentation, occlusion labeling, stereo and flow estimation’. Proc. European Conf. Computer Vision (ECCV), Zurich, Switzerland, September 2014, pp. 756771.
    30. 30)
      • 30. Philipp, K., Vladlen, K.: ‘Efficient inference in fully connected CRFs with Gaussian edge potentials’. Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS), Granada, Spain, December 2011, pp. 19.
    31. 31)
      • 31. Bo, L., Ren, X., Fox, D.: ‘Depth kernel descriptors for object recognition’. Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), San Francisco, CA, USA, September 2011, pp. 821826.
    32. 32)
      • 32. Shotton, J., Winn, J., Rother, C., et al: ‘Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation’. Proc. European Conf. Computer Vision (ECCV), Graz, Austria, May 2006, pp. 115.
    33. 33)
      • 33. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Large-scale semantic co-labeling of image sets’. Proc. IEEE Winter Conf. Applications of Computer Vision (WACV), Steamboat Springs CO., March 2014, pp. 501508.
    34. 34)
      • 34. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Exploiting large image sets for road scene parsing’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (9), pp. 24562465.
    35. 35)
      • 35. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Data-driven road detection’. Proc. IEEE Winter Conf. Applications of Computer Vision (WACV), Steamboat Springs, CO., March 2014, pp. 16.
    36. 36)
      • 36. Fritsch, J., Kuehnl, T., Geiger, A.: ‘A new performance measure and evaluation benchmark for road detection algorithms’. Proc. IEEE Int. Conf. Intelligent Transportation Systems (ITSC), Hague, Netherlands, October 2013, pp. 18.
    37. 37)
      • 37. Mendes, C., Frémont, V., Wolf, D.: ‘Vision-based road detection using contextual blocks’. arXiv:1509.01122, 2015. [Online]. Available at http://arxiv.org/abs/1509.01122, accessed September 2015.
    38. 38)
      • 38. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs’. arXiv:1606.00915. [Online] Available at https://arxiv.org/abs/1606.00915v2, accessed May 2017.
    39. 39)
      • 39. Redmon, J., Divvala, S., Girshick, R., et al: ‘You only look once: unified, real-time object detection’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, USA, June 2016, pp. 779788.
    40. 40)
      • 40. Liu, W., Anguelov, D., Erhan, D., et al: ‘SSD: single shot multibox detector’. European Conf. Computer Vision, Amsterdam, The Netherlands, October 2016, pp. 2137.
    41. 41)
      • 41. Iandola, F.N., Han, S., Moskewicz, M.W., et al: ‘SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size’. arXiv preprint arXiv:1602.07360, 2016. [Online] Available at https://arxiv.org/pdf/1602.07360.pdf, accessed November 2016.
    42. 42)
      • 42. Courbariaux, M., Bengio, Y.: ‘Binarynet: training deep neural networks with weights and activations constrained to+ 1 or−1’. arXiv:1602.02830. [Online] Available at https://arxiv.org/pdf/1602.02830.pdf, accessed November 2016.
    43. 43)
      • 43. Howard, A.G., Zhu, M., Chen, B., et al: ‘MobileNets: efficient convolutional neural networks for mobile vision applications’, arXiv preprint arXiv:1704.04861, 2017. [Online] Available at https://arxiv.org/pdf/1704.04861.pdf, accessed April 2017.
    44. 44)
      • 44. Romera, E., Alvarez, J.M., Bergasa, L.M., et al: ‘Efficient convnet for real-time semantic segmentation’. Proc. IEEE Intelligent Vehicles Symp. (IV), Los Angeles, CA, USA, July 2017, pp. 17891794.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2017.0266
Loading

Related content

content/journals/10.1049/iet-cvi.2017.0266
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address