Real-time detection of distracted driving based on deep learning

Real-time detection of distracted driving based on deep learning

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Driver distraction is a leading factor in car crashes. With a goal to reduce traffic accidents and improve transportation safety, this study proposes a driver distraction detection system which identifies various types of distractions through a camera observing the driver. An assisted driving testbed is developed for the purpose of creating realistic driving experiences and validating the distraction detection algorithms. The authors collected a dataset which consists of images of the drivers in both normal and distracted driving postures. Four deep convolutional neural networks including VGG-16, AlexNet, GoogleNet, and residual network are implemented and evaluated on an embedded graphic processing unit platform. In addition, they developed a conversational warning system that alerts the driver in real-time when he/she does not focus on the driving task. Experimental results show that the proposed approach outperforms the baseline one which has only 256 neurons in the fully-connected layers. Furthermore, the results indicate that the GoogleNet is the best model out of the four for distraction detection in the driving simulator testbed.


    1. 1)
      • 1. US Department of Transportation – National Highway Traffic Safety Administration: ‘Traffic safety facts’. Available at
    2. 2)
      • 2. US Department of Transportation – National Highway Traffic Safety Administration: ‘Distracted driving’. Available at:
    3. 3)
      • 3. Esurance: ‘3 types of distracted driving’, 2016. Available at:, accessed October 2017.
    4. 4)
      • 4. Just, M.A., Keller, T.A., Cynkar, J.: ‘A decrease in brain activation associated with driving when listening to someone speak’, Brain Res., 2008, 1205, pp. 7080.
    5. 5)
      • 5. Ameen, L.: ‘The 25 scariest texting and driving accident statistics’. Available at
    6. 6)
      • 6. Shiwu, L., Linhong, W., Zhifa, Y., et al: ‘An active driver fatigue identification technique using multiple physiological features’. Int. Conf. on Mechatronic Science, Electric Engineering and Computer (MEC), Jilin, China, 2011, pp. 733737.
    7. 7)
      • 7. Lal, S.K., Craig, A.: ‘Driver fatigue: electroencephalography and psychological assessment’, Psychophysiology, 2002, 39, (3), pp. 313321.
    8. 8)
      • 8. Jin, L., Niu, Q., Hou, H., et al: ‘Driver cognitive distraction detection using driving performance measures’, Discret. Dyn. Nat. Soc., 2012, 2012, Available at:
    9. 9)
      • 9. Ranney, T.A.: ‘Driver distraction: a review of the current state-of-knowledge’ (US Department of Transportation - National Highway Traffic Safety Administration, Washington DC, 2008).
    10. 10)
      • 10. Tabrizi, P.R., Zoroofi, R.A.: ‘Drowsiness detection based on brightness and numeral features of eye image’. Fifth Int. Conf. on Intelligent Information Hiding and Multimedia Signal Processing, Kyoto, Japan, 2009, pp. 13101313.
    11. 11)
      • 11. Farber, E., Foley, J., Scott, S.: ‘Visual attention design limits for its in-vehicle systems: the society of automotive engineers standard for limiting visual distraction while driving’. Transportation Research Board Annual General Meeting, Washington DC, USA, 2000, pp. 23.
    12. 12)
      • 12. Victor, T., Blomberg, O., Zelinsky, A.: ‘Automating the measurement of driver visual behaviours using passive stereo vision’. Proc. Int. Conf. on Series Vision Vehicles (VIV9), Brisbane, Queensland, Australia, 2001.
    13. 13)
      • 13. Kutila, M., Jokela, M., Markkula, G., et al: ‘Driver distraction detection with a camera vision system’. IEEE Int. Conf. on Image Processing, San Antonio, Texas, USA, 2007, vol. 6, pp. VI-201VI-204.
    14. 14)
      • 14. Fletcher, L., Zelinsky, A.: ‘Driver state monitoring to mitigate distraction’. Proc. Int. Conf. on the Distractions in Driving, Sydney, Australia, 2007, pp. 487523.
    15. 15)
      • 15. Azman, A., Meng, Q., Edirisinghe, E.: ‘Non-intrusive physiological measurement for driver cognitive distraction detection: eye and mouth movements’. Third Int. Conf. on Advanced Computer Theory and Engineering (ICACTE), Chengdu, China, 2010, vol. 3, pp. V3-595V3-599.
    16. 16)
      • 16. Park, S., Trivedi, M.: ‘Driver activity analysis for intelligent vehicles: issues and development framework’. IEEE Proc. Intelligent Vehicles Symp., Las Vegas, Nevada, USA, 2005, pp. 644649.
    17. 17)
      • 17. Pohl, J., Birk, W., Westervall, L.: ‘A driver-distraction-based lane-keeping assistance system’, Proc. Inst. Mech. Eng. I, J. Syst. Control Eng., 2007, 221, (4), pp. 541552.
    18. 18)
      • 18. Kircher, K., Ahlstrom, C., Kircher, A.: ‘Comparison of two eye-gaze based realtime driver distraction detection algorithms in a small-scale field operational test’. Proc. Fifth Int. Symp. on Human Factors in Driver Assessment, Training and Vehicle Design, Big Sky, Montana, USA, 2009, pp. 1623.
    19. 19)
      • 19. Murphy-Chutorian, E., Doshi, A., Trivedi, M. M.: ‘Head pose estimation for driver assistance systems: a robust algorithm and experimental evaluation’. IEEE Intelligent Transportation Systems Conf., Seattle, Washington, USA, 2007, pp. 709714.
    20. 20)
      • 20. Bergasa, L.M., Nuevo, J., Sotelo, M.A., et al: ‘Real-time system for monitoring driver vigilance’, IEEE Trans. Intell. Transp. Syst., 2006, 7, (1), pp. 6377.
    21. 21)
      • 21. Ji, Q., Lan, P., Looney, C.: ‘A probabilistic framework for modeling and real-time monitoring human fatigue’, IEEE Trans. Syst. Man Cybern. A, Syst. Humans, 2006, 36, (5), pp. 862875.
    22. 22)
      • 22. Craye, C., Karray, F.: ‘Multi-distributions particle filter for eye tracking inside a vehicle’, Image Anal. Recognit., 2013, 6, pp. 407416.
    23. 23)
      • 23. Ji, Q., Zhu, Z., Lan, P.: ‘Real-time nonintrusive monitoring and prediction of driver fatigue’, IEEE Trans. Veh. Technol., 2004, 53, (4), pp. 10521068.
    24. 24)
      • 24. Craye, C., Karray, F.: ‘Driver distraction detection and recognition using RGB-D sensor’, CoRR, 2015, abs/1502.00250. Available at:
    25. 25)
      • 25. Liang, Y., Reyes, M.L., Lee, J.D.: ‘Real-time detection of driver cognitive distraction using support vector machines’, IEEE Trans. Intell. Transp. Syst., 2007, 8, (2), pp. 340350.
    26. 26)
      • 26. Gu, H., Ji, Q.: ‘Facial event classification with task oriented dynamic Bayesian network’. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR, Washington, DC, USA, 2004, vol. 2, pp. II-870II-875.
    27. 27)
      • 27. Eskandarian, A., Sayed, R.: ‘Driving simulator experiment: detecting driver fatigue by monitoring eye and steering activity’. Proc. Annual Intelligent Vehicles Systems Symp., Traverse City, Michigan, USA, 2003.
    28. 28)
      • 28. Eskandarian, A., Sayed, R.: ‘Analysis of driver impairment, fatigue, and drowsiness and an unobtrusive vehicle-based detection scheme’. Proc. 1st Int. Conf. on Traffic Accidents, Tehran, Iran, 2005, pp. 3549.
    29. 29)
      • 29. State Farm Corporate: ‘State farm distracted driver detection’. Available at
    30. 30)
      • 30. Colbran, S., Cen, K., Luo, D.: ‘Classification of driver dis- traction’ (Stanford University, Stanford, CA, 2016). Available at:
    31. 31)
      • 31. Okon, O.D., Meng, L.: ‘Detecting distracted driving with deep learning’. Interactive Collaborative Robotics, Hatfield, United Kingdom, 2017, pp. 170179.
    32. 32)
      • 32. Abouelnaga, Y., Eraqi, H.M., Moustafa, M.N.: ‘Real-time distracted driver posture classification’, arXiv preprint arXiv:170609498, 2017.
    33. 33)
      • 33. Lőrincz, A., Csákvári, M., Fóthi, Á., et al: ‘Cognitive deep machine can train itself’, arXiv preprint arXiv:161200745, 2016.
    34. 34)
      • 34. Choi, I.H., Hong, S.K., Kim, Y.G.: ‘Real-time categorization of driver's gaze zone using the deep learning techniques’. Int. Conf. on Big Data and Smart Computing (BigComp), Jeongseon, Republic of Korea, 2016, pp. 143148.
    35. 35)
      • 35. Venturelli, M., Borghi, G., Vezzani, R., et al: ‘Deep head pose estimation from depth data for in-car automotive applications’, arXiv preprint arXiv:170301883, 2017.
    36. 36)
      • 36. Hssayeni, M.D., Saxena, S., Ptucha, R., et al: ‘Distracted driver detection: deep learning vs. handcrafted features’, Electron. Imaging, 2017, 7, (10), p. 20–.
    37. 37)
      • 37. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:14091556, 2014.
    38. 38)
      • 38. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘ImageNet classification with deep convolutional neural networks’. Adv. Neural. Inf. Process. Syst., 2012, 1, pp. 10971105.
    39. 39)
      • 39. Szegedy, C., Liu, W., Jia, Y., et al: ‘Going deeper with convolutions’. Proc. IEEE Conf. on computer vision and pattern recognition, Boston, Massachusetts, USA, 2015, pp. 19.
    40. 40)
      • 40. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Conf. on computer vision and pattern recognition, Las Vegas, Nevada, USA, 2016, pp. 770778.
    41. 41)
      • 41. Shin, H.C., Roth, H.R., Gao, M., et al: ‘Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning’, IEEE Trans. Med. Imaging, 2016, 35, (5), pp. 12851298.
    42. 42)
      • 42. Carnetsoft Inc.: ‘Research driving simulator’. Available at
    43. 43)
      • 43. NVIDIA: ‘Embedded systems’. Available at
    44. 44)
      • 44. FriendlyARM: ‘Nanopi M3’. Available at
    45. 45)
      • 45. Srivastava, N., Hinton, G.E., Krizhevsky, A., et al: ‘Dropout: a simple way to prevent neural networks from overfitting’, J. Mach. Learn. Res., 2014, 15, (1), pp. 19291958.
    46. 46)
      • 46. Bottou, L.: ‘Large-scale machine learning with stochastic gradient descent’. Proc. COMPSTAT, 2010, pp. 177186.

Related content

This is a required field
Please enter a valid email address