http://iet.metastore.ingenta.com
1887

Combining 2D and 3D features to improve road detection based on stereo cameras

Combining 2D and 3D features to improve road detection based on stereo cameras

For access to this article, please select a purchase option:

Buy eFirst article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
— Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Road detection is a fundamental component of autonomous driving systems since it provides valid space and candidate regions of objects for driving decision. The core of road detection methods is extracting effective and discriminative features. Since two-dimensional (2D) and 3D features are complementary, the authors propose a robust multi-feature combination and optimisation framework for stereo image pairs, called Feature++. First, several 2D and 3D features such as Gabor and plane are, respectively, extracted after the generation of 2D super-pixel and a 3D depth image from stereo matching. Second, the combined features are fed into a three-layer shallow neural network classifier to decide whether a super-pixel is road region or not. Finally, the classified results are further refined using fully connected conditional random field (CRF), taking the content information into consideration. We extensively evaluate the performance of four 2D features, four 3D features, and their combinations. Experiments conducted on the KITTI ROAD benchmark show that (i) the combinations of 2D and 3D features greatly improve the road detection performance and (ii) using CRF as a refinement step is necessary. Overall, their proposed ‘Feature + +’ method outperforms most manually designed features, and is comparable with state-of-the-art methods that are based on deep learning methods.

References

    1. 1)
      • J. Gao , Q. Wang , Y. Yuan .
        1. Gao, J., Wang, Q., Yuan, Y.: ‘Embedding structured contour and location prior in siamesed fully convolutional networks for road detection’. Proc. IEEE Int. Conf. Robotics and Automation (ICRA), Singapore, May 2017, pp. 17.
        . Proc. IEEE Int. Conf. Robotics and Automation (ICRA) , 1 - 7
    2. 2)
      • M. Kocamaz , L. Navarro-Serment , M. Hebert .
        2. Kocamaz, M., Navarro-Serment, L., Hebert, M.: ‘Map-supervised road detection’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Gothenburg, Sweden, June 2016, pp. 16.
        . Proc. IEEE Int. Conf. Intelligent Vehicles Symp. , 1 - 6
    3. 3)
      • G. Oliveira , W. Burgard , T. Brox .
        3. Oliveira, G., Burgard, W., Brox, T.: ‘Efficient deep methods for monocular road segmentation’. Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), Daejeon, South Korea, October 2016, pp. 18.
        . Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS) , 1 - 8
    4. 4)
      • J. Muñoz-Bulnes , C. Fernandez , I. Parra .
        4. Muñoz-Bulnes, J., Fernandez, C., Parra, I., et al: ‘Deep fully convolutional networks with random data augmentation for enhanced generalization in road detection’. Proc. IEEE Int. Conf. Intelligent Transportation Systems Workshop on Deep Learning for Autonomous Driving, Yokohama, Japan, October 2017, pp. 16.
        . Proc. IEEE Int. Conf. Intelligent Transportation Systems Workshop on Deep Learning for Autonomous Driving , 1 - 6
    5. 5)
      • X. Chen , K. Kundu , Y. Zhu .
        5. Chen, X., Kundu, K., Zhu, Y., et al: ‘3D object proposals for accurate object class detection’. Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS), Montréal, Canada, December 2015, pp. 424432.
        . Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS) , 424 - 432
    6. 6)
      • D. Gabor .
        6. Gabor, D.: ‘Theory of communication – part 1: the analysis of information’, J. Inst. Electr. Eng. III, 2010, 93, (26), pp. 429441.
        . J. Inst. Electr. Eng. III , 26 , 429 - 441
    7. 7)
      • L. Bo , X. Ren , D. Fox .
        7. Bo, L., Ren, X., Fox, D.: ‘Kernel descriptors for visual recognition’. Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS), Vancouver, Canada, December 2010, pp. 244252.
        . Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS) , 244 - 252
    8. 8)
      • W.L. He , G.R. Cai , Z. Zhong .
        8. He, W.L., Cai, G.R., Zhong, Z., et al: ‘Feature++: cross dimension feature fusion for road detection’. Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP), New Orleans, USA, March 2017, pp. 16.
        . Proc. IEEE Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP) , 1 - 6
    9. 9)
      • J. Alvarez , A.M. Lopez .
        9. Alvarez, J., Lopez, A.M.: ‘Road detection based on illuminant invariance’, IEEE Trans. Intell. Transp. Syst., 2011, 12, (1), pp. 184193.
        . IEEE Trans. Intell. Transp. Syst. , 1 , 184 - 193
    10. 10)
      • B. Wang , V. Frémont , S. Rodriguez .
        10. Wang, B., Frémont, V., Rodriguez, S.: ‘Color-based road detection and its evaluation on the KITTI road benchmark’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Dearborn, MI, June 2014, pp. 3136.
        . Proc. IEEE Int. Conf. Intelligent Vehicles Symp. , 31 - 36
    11. 11)
      • J.M. Alvarez , M. Salzmann , N. Barnes .
        11. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Learning appearance models for road detection’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Gold Coast, QLD, Australia, June 2013, pp. 423429.
        . Proc. IEEE Int. Conf. Intelligent Vehicles Symp. , 423 - 429
    12. 12)
      • J.M. Alvarez , T. Gevers , Y. LeCun .
        12. Alvarez, J.M., Gevers, T., LeCun, Y., et al: ‘Road scene segmentation from a single image’. Proc. European Conf. Computer Vision (ECCV), Firenze, Italy, October 2012, pp. 376389.
        . Proc. European Conf. Computer Vision (ECCV) , 376 - 389
    13. 13)
      • C. Rasmussen .
        13. Rasmussen, C.: ‘Grouping dominant orientations for ill-structured road following’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Washington DC, USA, June 2004, pp. 18.
        . Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR) , 1 - 8
    14. 14)
      • H. Kong , J.-Y. Audibert , J. Ponce .
        14. Kong, H., Audibert, J.-Y., Ponce, J.: ‘Vanishing point detection for road detection’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Miami, FL, June 2009, pp. 96103.
        . Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR) , 96 - 103
    15. 15)
      • J. Fritsch , T. Kuehnl , F. Kummert .
        15. Fritsch, J., Kuehnl, T., Kummert, F.: ‘Monocular road terrain detection by combining visual and spatial information’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (4), pp. 15861596.
        . IEEE Trans. Intell. Transp. Syst. , 4 , 1586 - 1596
    16. 16)
      • R. Mohan .
        16. Mohan, R.: ‘Deep deconvolutional networks for scene parsing’. arXiv: Machine Learning, November 2014. Available at https://arxiv.org/pdf/1411.4101.pdf, accessed November 2014.
        .
    17. 17)
      • H. Dahlkamp , A. Kaehler , D. Stavens .
        17. Dahlkamp, H., Kaehler, A., Stavens, D., et al: ‘Self-supervised monocular road detection in desert terrain’. Proc. IEEE Int. Conf. Robotics Science Systems, Pennsylvania, USA, August 2006, pp. 17.
        . Proc. IEEE Int. Conf. Robotics Science Systems , 1 - 7
    18. 18)
      • Y. Alon , A. Ferencz , A. Shashua .
        18. Alon, Y., Ferencz, A., Shashua, A.: ‘Off-road path following using region classification and geometric projection constraints’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, June 2006, pp. 689696.
        . Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition (CVPR) , 689 - 696
    19. 19)
      • L. Xiao , B. Dai , D. Liu .
        19. Xiao, L., Dai, B., Liu, D., et al: ‘CRF based road detection with multi-sensor fusion’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp. (IV), Seoul, South Korea, June 2015, pp. 192198.
        . Proc. IEEE Int. Conf. Intelligent Vehicles Symp. (IV) , 192 - 198
    20. 20)
      • G. Vitor , D. Lima , A. Victorino .
        20. Vitor, G., Lima, D., Victorino, A., et al: ‘A 2D/3D vision based approach applied to road detection in urban environments’, IEEE. Intell. Veh. Symp. (IV), 2013, 36, (1), pp. 952957.
        . IEEE. Intell. Veh. Symp. (IV) , 1 , 952 - 957
    21. 21)
      • S. Thrun , M. Montemerlo , H. Dahlkamp . (2007)
        21. Thrun, S., Montemerlo, M., Dahlkamp, H., et al: ‘Stanley: the robot that won the DARPA grand challenge’ (Stanford University, Stanford, CA, 2007), pp. 143.
        .
    22. 22)
      • F. Moosmann , O. Pink , C. Stiller .
        22. Moosmann, F., Pink, O., Stiller, C.: ‘Segmentation of 3d LiDAR data in non-flat urban environments using a local convexity criterion’. Proc. IEEE Int. Conf. Intelligent Vehicles Symp., Xi'an, China, June 2009, pp. 215220.
        . Proc. IEEE Int. Conf. Intelligent Vehicles Symp. , 215 - 220
    23. 23)
      • T. Chen , B. Dai , R. Wang .
        23. Chen, T., Dai, B., Wang, R., et al: ‘Gaussian-process-based real time ground segmentation for autonomous land vehicles’, J. Intell. Robot. Syst. (JINT), 2014, 76, (3), pp. 563582.
        . J. Intell. Robot. Syst. (JINT) , 3 , 563 - 582
    24. 24)
      • R. Labayrade , D. Aubert , J.-P. Tarel .
        24. Labayrade, R., Aubert, D., Tarel, J.-P.: ‘Real time obstacle detection in stereovision on non-flat road geometry through ‘v-disparity’ representation’. Proc. IEEE Int. Conf. Intelligent Vehicle Symp., Versailles, France, June 2002, pp. 646651.
        . Proc. IEEE Int. Conf. Intelligent Vehicle Symp. , 646 - 651
    25. 25)
      • H. Badino , U. Franke , R. Mester .
        25. Badino, H., Franke, U., Mester, R.: ‘Free space computation using stochastic occupancy grids and dynamic programming’. Proc. IEEE Int. Conf. Computer Vision (ICCV) Workshop on Dynamical Vision, Rio de Janeiro, Brazil, October 2007, pp. 18.
        . Proc. IEEE Int. Conf. Computer Vision (ICCV) Workshop on Dynamical Vision , 1 - 8
    26. 26)
      • F. Oniga , S. Nedevschi .
        26. Oniga, F., Nedevschi, S.: ‘Processing dense stereo data using elevation maps: road surface, traffic isle, and obstacle detection’, IEEE Trans. Veh. Technol, 2010, 59, (3), pp. 11721182.
        . IEEE Trans. Veh. Technol , 3 , 1172 - 1182
    27. 27)
      • J.K. Suhr , H.G. Jung .
        27. Suhr, J.K., Jung, H.G.: ‘Dense stereo-based robust vertical road profile estimation using Hough transform and dynamic programming’, IEEE Trans. Intell. Transp. Syst., 2015, 16, (3), pp. 15281536.
        . IEEE Trans. Intell. Transp. Syst. , 3 , 1528 - 1536
    28. 28)
      • A.G. Ramakrishnan , R.S. Kumar , H.V. Raghu .
        28. Ramakrishnan, A.G., Kumar, R.S., Raghu, H.V.: ‘Neural network-based segmentation of textures using Gabor features’. Proc. IEEE Int. Workshop on Neural Networks for Signal Processing, Martigny, Switzerland, September 2002, pp. 365374.
        . Proc. IEEE Int. Workshop on Neural Networks for Signal Processing , 365 - 374
    29. 29)
      • K. Yamaguchi , D. McAllester , R. Urtasun .
        29. Yamaguchi, K., McAllester, D., Urtasun, R.: ‘Efficient joint segmentation, occlusion labeling, stereo and flow estimation’. Proc. European Conf. Computer Vision (ECCV), Zurich, Switzerland, September 2014, pp. 756771.
        . Proc. European Conf. Computer Vision (ECCV) , 756 - 771
    30. 30)
      • K. Philipp , K. Vladlen .
        30. Philipp, K., Vladlen, K.: ‘Efficient inference in fully connected CRFs with Gaussian edge potentials’. Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS), Granada, Spain, December 2011, pp. 19.
        . Proc. IEEE Int. Conf. Neural Information Processing Systems (NIPS) , 1 - 9
    31. 31)
      • L. Bo , X. Ren , D. Fox .
        31. Bo, L., Ren, X., Fox, D.: ‘Depth kernel descriptors for object recognition’. Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS), San Francisco, CA, USA, September 2011, pp. 821826.
        . Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS) , 821 - 826
    32. 32)
      • J. Shotton , J. Winn , C. Rother .
        32. Shotton, J., Winn, J., Rother, C., et al: ‘Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation’. Proc. European Conf. Computer Vision (ECCV), Graz, Austria, May 2006, pp. 115.
        . Proc. European Conf. Computer Vision (ECCV) , 1 - 15
    33. 33)
      • J.M. Alvarez , M. Salzmann , N. Barnes .
        33. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Large-scale semantic co-labeling of image sets’. Proc. IEEE Winter Conf. Applications of Computer Vision (WACV), Steamboat Springs CO., March 2014, pp. 501508.
        . Proc. IEEE Winter Conf. Applications of Computer Vision (WACV) , 501 - 508
    34. 34)
      • J.M. Alvarez , M. Salzmann , N. Barnes .
        34. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Exploiting large image sets for road scene parsing’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (9), pp. 24562465.
        . IEEE Trans. Intell. Transp. Syst. , 9 , 2456 - 2465
    35. 35)
      • J.M. Alvarez , M. Salzmann , N. Barnes .
        35. Alvarez, J.M., Salzmann, M., Barnes, N.: ‘Data-driven road detection’. Proc. IEEE Winter Conf. Applications of Computer Vision (WACV), Steamboat Springs, CO., March 2014, pp. 16.
        . Proc. IEEE Winter Conf. Applications of Computer Vision (WACV) , 1 - 6
    36. 36)
      • J. Fritsch , T. Kuehnl , A. Geiger .
        36. Fritsch, J., Kuehnl, T., Geiger, A.: ‘A new performance measure and evaluation benchmark for road detection algorithms’. Proc. IEEE Int. Conf. Intelligent Transportation Systems (ITSC), Hague, Netherlands, October 2013, pp. 18.
        . Proc. IEEE Int. Conf. Intelligent Transportation Systems (ITSC) , 1 - 8
    37. 37)
      • C. Mendes , V. Frémont , D. Wolf .
        37. Mendes, C., Frémont, V., Wolf, D.: ‘Vision-based road detection using contextual blocks’. arXiv:1509.01122, 2015. [Online]. Available at http://arxiv.org/abs/1509.01122, accessed September 2015.
        .
    38. 38)
      • L.C. Chen , G. Papandreou , I. Kokkinos .
        38. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs’. arXiv:1606.00915. [Online] Available at https://arxiv.org/abs/1606.00915v2, accessed May 2017.
        .
    39. 39)
      • J. Redmon , S. Divvala , R. Girshick .
        39. Redmon, J., Divvala, S., Girshick, R., et al: ‘You only look once: unified, real-time object detection’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, USA, June 2016, pp. 779788.
        . Proc. IEEE Conf. Computer Vision and Pattern Recognition , 779 - 788
    40. 40)
      • W. Liu , D. Anguelov , D. Erhan .
        40. Liu, W., Anguelov, D., Erhan, D., et al: ‘SSD: single shot multibox detector’. European Conf. Computer Vision, Amsterdam, The Netherlands, October 2016, pp. 2137.
        . European Conf. Computer Vision , 21 - 37
    41. 41)
      • F.N. Iandola , S. Han , M.W. Moskewicz .
        41. Iandola, F.N., Han, S., Moskewicz, M.W., et al: ‘SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size’. arXiv preprint arXiv:1602.07360, 2016. [Online] Available at https://arxiv.org/pdf/1602.07360.pdf, accessed November 2016.
        .
    42. 42)
      • M. Courbariaux , Y. Bengio .
        42. Courbariaux, M., Bengio, Y.: ‘Binarynet: training deep neural networks with weights and activations constrained to+ 1 or−1’. arXiv:1602.02830. [Online] Available at https://arxiv.org/pdf/1602.02830.pdf, accessed November 2016.
        .
    43. 43)
      • A.G. Howard , M. Zhu , B. Chen .
        43. Howard, A.G., Zhu, M., Chen, B., et al: ‘MobileNets: efficient convolutional neural networks for mobile vision applications’, arXiv preprint arXiv:1704.04861, 2017. [Online] Available at https://arxiv.org/pdf/1704.04861.pdf, accessed April 2017.
        .
    44. 44)
      • E. Romera , J.M. Alvarez , L.M. Bergasa .
        44. Romera, E., Alvarez, J.M., Bergasa, L.M., et al: ‘Efficient convnet for real-time semantic segmentation’. Proc. IEEE Intelligent Vehicles Symp. (IV), Los Angeles, CA, USA, July 2017, pp. 17891794.
        . Proc. IEEE Intelligent Vehicles Symp. (IV) , 1789 - 1794
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2017.0266
Loading

Related content

content/journals/10.1049/iet-cvi.2017.0266
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address