Lateral distance detection model based on convolutional neural network

Lateral distance detection model based on convolutional neural network

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

To evaluate the performance of the advanced driver assistant systems, such as lane departure warning systems (LDWs) and lane keeping assist systems (LKAs), a deep learning model is proposed to estimate the lateral distance between the vehicle and lane boundaries. The training of a deep learning model requires a large number of label images, but the generation of label images is time consuming and boring. Therefore, an improved image quilting algorithm based on a convolutional neural network is proposed. A lot of lane and asphalt pavement images can be synthesised using fewer images of a real road scene. Moreover, an algorithm that aims to automatically generate label images using lane and asphalt pavement images to satisfy the distribution of real scenes is proposed. Experimental results showed that the generated label images can be used to train a deep learning model, and the lateral distance can be estimated with a sub-centimetre precision, which can provide an effective benchmark for the road test of LDWs, LKAs and other driving assistant systems.


    1. 1)
      • 1. Huang, Y., Wang, H., Khajepour, A., et al: ‘Model predictive control power management strategies for HEVs: a review’, J. Power Sources, 2017, 341, pp. 91106.
    2. 2)
      • 2. Huang, Y., Khajepour, A., Ding, H., et al: ‘An energy-saving set-point optimizer with a sliding mode controller for automotive air-conditioning/refrigeration systems’, Appl. Energy, 2017, 188, pp. 576585.
    3. 3)
      • 3. Tang, X., Yang, W., Hu, X., et al: ‘A novel simplified model for torsional vibration analysis of a series-parallel hybrid electric vehicle’, Mech. Syst. Signal Process., 2017, 85, pp. 329338.
    4. 4)
      • 4. Wang, J., Dai, M., Yin, G., et al: ‘Output-feedback robust control for vehicle path tracking considering different human drivers’ characteristics’, Mechatronics, 2017, doi: 10.1016/j.mechatronics.2017.05.001.
    5. 5)
      • 5. Qin, Y., He, C., Shao, X., et al: ‘Vibration mitigation for in-wheel switched reluctance motor driven electric vehicle with dynamic vibration absorbing structures’, J. Sound Vib., 2018, 419, pp. 249267.
    6. 6)
      • 6. Wang, H., Huang, Y., Khajepour, A., et al: ‘A novel energy management for hybrid off-road vehicles without future driving cycles as a priori’, Energy, 2017, 133, pp. 929940.
    7. 7)
      • 7. Zou, C., Hu, X., Wei, Z., et al: ‘Electrothermal dynamics-conscious lithium-ion battery cell-level charging management via state-monitored predictive control’, Energy, 2017, 141, pp. 250259.
    8. 8)
      • 8. Tang, X., Hu, X., Yang, W., et al: ‘Novel torsional vibration modeling and assessment of a power-split hybrid electric vehicle equipped with a dual mass flywheel’, IEEE Trans. Veh. Technol., 2017, 67, pp. 19902000, doi:10.1109/TVT.2017.2769084.
    9. 9)
      • 9. Lee, H., Jeong, S., Lee, J.: ‘Robust detection system of illegal lane changes based on tracking of feature points’, IET Intell. Transp. Syst., 2013, 7, (1), pp. 2027.
    10. 10)
      • 10. Lorenz, P., Schäufele, B., Sawade, O., et al: ‘Recursive state estimation for lane detection using a fusion of cooperative and map based data’. IEEE 18th Int. Conf. on Intelligent Transportation Systems, Las Palmas de Gran Canaria Canary Islands, Spain, 2015, pp. 21802185.
    11. 11)
      • 11. Borkar, A., Hayes, M., Smith, M.T.: ‘A novel lane detection system with efficient ground truth generation’, IEEE Trans. Intell. Transp. Syst., 2012, 13, (1), pp. 365374.
    12. 12)
      • 12. Al-Sarraf, A., Shin, B.S., Xu, Z., et al: ‘Ground truth and performance evaluation of lane border detection’. Int. Conf. on Computer Vision and Graphics, Warsaw, Poland, September 2014, pp. 6674.
    13. 13)
      • 13. Hata, A.Y., Wolf, D.F.: ‘Feature detection for vehicle localization in urban environments using a multilayer LIDAR’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (2), pp. 420429.
    14. 14)
      • 14. Ma, D., Luo, X., Li, W., et al: ‘Traffic demand estimation for lane groups at signal-controlled intersections using travel times from video-imaging detectors’, IET Intell. Transp. Syst., 2017, 11, (4), pp. 222229.
    15. 15)
      • 15. Baili, J., Marzougui, M., Sboui, A., et al: ‘Lane Departure detection using image processing techniques’. 2017 2nd IEEE Int. Conf. on Anti-Cyber Crimes (ICACC, 2017), Abha, Saudi Arabia, March 2017, pp. 238241.
    16. 16)
      • 16. Mankar, S.J., Demde, M., Sharma, P.:Design of computer vision intelligent system for lane detection’. IEEE Int. Conf. on Green Engineering and Technologies (IC-GET), Coimbatore, India, November 2016, pp. 13.
    17. 17)
      • 17. El Hajjouji, I., El Mourabit, A., Asrih, Z., et al: ‘FPGA based real-time lane detection and tracking implementation’. 2016 IEEE Int. Conf. on Electrical and Information Technologies (ICEIT, 2016), Tangiers, Morocco, May 2016, pp. 186190.
    18. 18)
      • 18. Gruyer, D., Belaroussi, R., Revilloud, M.: ‘Accurate lateral positioning from map data and road marking detection’, Expert Syst. Appl., 2016, 43, pp. 18.
    19. 19)
      • 19. Li, J., Mei, X., Prokhorov, D., et al: ‘Deep neural network for structural prediction and lane detection in traffic scene’, IEEE Trans. Neural Netw. Learn. Syst., 2017, 28, (3), pp. 690703.
    20. 20)
      • 20. Brust, C.A., Sickert, S., Simon, M., et al: ‘Convolutional patch networks with spatial prior for road detection and urban scene understanding’, arXiv preprint arXiv:1502.06344, 2015.
    21. 21)
      • 21. Kim, J., Lee, M.:Robust lane detection based on convolutional neural network and random sample consensus’. Int. Conf. on Neural Information Processing, Sarawak, Malaysia, November 2014, pp. 454461.
    22. 22)
      • 22. Qin, Y., Langari, R., Wang, Z., et al: ‘Road excitation classification for semi-active suspension system with deep neural networks’, J. Intell. Fuzzy Syst., 2017, 33, (3), pp. 19071918.
    23. 23)
      • 23. Revilloud, M., Gruyer, D., Rahal, M.C.: ‘A new multi-agent approach for lane detection and tracking’. IEEE Int. Conf. on Robotics and Automation (ICRA), Stockholm, Sweden, May 2016, pp. 31473153.
    24. 24)
      • 24. Gurghian, A., Koduri, T., Bailur, S.V., et al:DeepLanes: end-to-end lane position estimation using deep neural networks’. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, Nevada, July 2016, pp. 3845.
    25. 25)
      • 25. Marc, R., Dominique, G., Evangeline, P.: ‘Generator of road marking textures and associated ground truth applied to the evaluation of road marking detection’. 2012 15th Int. IEEE Conf. on Intelligent Transportation Systems, Anchorage, AK, USA, 2012, pp. 933938.
    26. 26)
      • 26. Santana, E., Hotz, G.: ‘Learning a driving simulator’, arXiv preprint arXiv:1608.01230, 2016.
    27. 27)
      • 27. Efros, A.A., Freeman, W.T.: ‘Image quilting for texture synthesis and transfer’. 28th Annual Conf. on Computer Graphics and Interactive Techniques, ACM, Los Angeles, CA, USA, August 2001, pp. 341346.
    28. 28)
      • 28. Galerne, B., Gousseau, Y., Morel, J.M.: ‘Random phase textures: theory and synthesis’, IEEE Trans. Image Process., 2011, 20, (1), pp. 257267.
    29. 29)
      • 29. Long, J., Mould, D.: ‘Improved image quilting’. Graphics Interface ACM, Montreal, Canada, 2007, pp. 257264.
    30. 30)
      • 30. Van, Q.N., Yoon, M., Che, W., et al: ‘A study on real time integrated lane detection and vehicle tracking method with side-mirror cameras’. IEEE 14th Int. Workshop on Advanced Motion Control (AMC), Auckland, New Zealand, April 2016, pp. 346352.
    31. 31)
      • 31. Shelhamer, E., Long, J., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (4), pp. 640651.
    32. 32)
      • 32. Shin, H.C., Roth, H.R., Gao, M., et al: ‘Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning’, IEEE Trans. Med. Imaging, 2016, 35, (5), pp. 12851298..
    33. 33)
      • 33. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘ImageNet classification with deep convolutional neural networks’. Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 2012, pp. 10971105.
    34. 34)
      • 34. Babenko, A., Slesarev, A., Chigorin, A., et al: ‘Neural codes for image retrieval’. European Conf. on Computer Vision, Zurich, Switzerland, September 2014, pp. 584599.
    35. 35)
      • 35. Backenroth, D., Goldsmith, J., Harran, M.D., et al: ‘Modeling motor learning using heteroskedastic functional principal components analysis’, J. Am. Stat. Assoc., 2017, doi: 10.1080/01621459.2017.1379403.
    36. 36)
      • 36. Raad, L., Galerne, B.: ‘Efros and Freeman image quilting algorithm for texture synthesis’, Image Process. On Line, 2017, 7, pp. 122.
    37. 37)
      • 37. Chung, K.W., Chan, H.S.Y., Wang, B.N.: ‘Automatic generation of nonperiodic patterns from dynamical systems’, Chaos, Solitons Fract., 2004, 19, (5), pp. 11771187.
    38. 38)
      • 38. Alvarez, J.M., Lopez, A., Baldrich, R.: ‘Illuminant-invariant model-based road segmentation’. IEEE Intelligent Vehicles Symp., Netherlands, June 2008, pp. 11751180.
    39. 39)
      • 39. Lin, K., Yang, H.F., Hsiao, J.H., et al: ‘Deep learning of binary hash codes for fast image retrieval’. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston, Massachusetts, June 2015, pp. 2735.
    40. 40)
      • 40. Zheng, S., Jayasumana, S., Romera-Paredes, B., et al: ‘Conditional random fields as recurrent neural networks’. IEEE Int. Conf. on Computer Vision (ICCV), Chile, December 2015, pp. 15291537.

Related content

This is a required field
Please enter a valid email address