Your browser does not support JavaScript!

access icon free Real-time detection of distracted driving based on deep learning

Driver distraction is a leading factor in car crashes. With a goal to reduce traffic accidents and improve transportation safety, this study proposes a driver distraction detection system which identifies various types of distractions through a camera observing the driver. An assisted driving testbed is developed for the purpose of creating realistic driving experiences and validating the distraction detection algorithms. The authors collected a dataset which consists of images of the drivers in both normal and distracted driving postures. Four deep convolutional neural networks including VGG-16, AlexNet, GoogleNet, and residual network are implemented and evaluated on an embedded graphic processing unit platform. In addition, they developed a conversational warning system that alerts the driver in real-time when he/she does not focus on the driving task. Experimental results show that the proposed approach outperforms the baseline one which has only 256 neurons in the fully-connected layers. Furthermore, the results indicate that the GoogleNet is the best model out of the four for distraction detection in the driving simulator testbed.


    1. 1)
      • 25. Liang, Y., Reyes, M.L., Lee, J.D.: ‘Real-time detection of driver cognitive distraction using support vector machines’, IEEE Trans. Intell. Transp. Syst., 2007, 8, (2), pp. 340350.
    2. 2)
      • 33. Lőrincz, A., Csákvári, M., Fóthi, Á., et al: ‘Cognitive deep machine can train itself’, arXiv preprint arXiv:161200745, 2016.
    3. 3)
      • 10. Tabrizi, P.R., Zoroofi, R.A.: ‘Drowsiness detection based on brightness and numeral features of eye image’. Fifth Int. Conf. on Intelligent Information Hiding and Multimedia Signal Processing, Kyoto, Japan, 2009, pp. 13101313.
    4. 4)
      • 5. Ameen, L.: ‘The 25 scariest texting and driving accident statistics’. Available at
    5. 5)
      • 40. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Conf. on computer vision and pattern recognition, Las Vegas, Nevada, USA, 2016, pp. 770778.
    6. 6)
      • 45. Srivastava, N., Hinton, G.E., Krizhevsky, A., et al: ‘Dropout: a simple way to prevent neural networks from overfitting’, J. Mach. Learn. Res., 2014, 15, (1), pp. 19291958.
    7. 7)
      • 8. Jin, L., Niu, Q., Hou, H., et al: ‘Driver cognitive distraction detection using driving performance measures’, Discret. Dyn. Nat. Soc., 2012, 2012, Available at:
    8. 8)
      • 23. Ji, Q., Zhu, Z., Lan, P.: ‘Real-time nonintrusive monitoring and prediction of driver fatigue’, IEEE Trans. Veh. Technol., 2004, 53, (4), pp. 10521068.
    9. 9)
      • 42. Carnetsoft Inc.: ‘Research driving simulator’. Available at
    10. 10)
      • 15. Azman, A., Meng, Q., Edirisinghe, E.: ‘Non-intrusive physiological measurement for driver cognitive distraction detection: eye and mouth movements’. Third Int. Conf. on Advanced Computer Theory and Engineering (ICACTE), Chengdu, China, 2010, vol. 3, pp. V3-595V3-599.
    11. 11)
      • 4. Just, M.A., Keller, T.A., Cynkar, J.: ‘A decrease in brain activation associated with driving when listening to someone speak’, Brain Res., 2008, 1205, pp. 7080.
    12. 12)
      • 28. Eskandarian, A., Sayed, R.: ‘Analysis of driver impairment, fatigue, and drowsiness and an unobtrusive vehicle-based detection scheme’. Proc. 1st Int. Conf. on Traffic Accidents, Tehran, Iran, 2005, pp. 3549.
    13. 13)
      • 29. State Farm Corporate: ‘State farm distracted driver detection’. Available at
    14. 14)
      • 20. Bergasa, L.M., Nuevo, J., Sotelo, M.A., et al: ‘Real-time system for monitoring driver vigilance’, IEEE Trans. Intell. Transp. Syst., 2006, 7, (1), pp. 6377.
    15. 15)
      • 1. US Department of Transportation – National Highway Traffic Safety Administration: ‘Traffic safety facts’. Available at
    16. 16)
      • 41. Shin, H.C., Roth, H.R., Gao, M., et al: ‘Deep convolutional neural networks for computer-aided detection: CNN architectures, dataset characteristics and transfer learning’, IEEE Trans. Med. Imaging, 2016, 35, (5), pp. 12851298.
    17. 17)
      • 16. Park, S., Trivedi, M.: ‘Driver activity analysis for intelligent vehicles: issues and development framework’. IEEE Proc. Intelligent Vehicles Symp., Las Vegas, Nevada, USA, 2005, pp. 644649.
    18. 18)
      • 9. Ranney, T.A.: ‘Driver distraction: a review of the current state-of-knowledge’ (US Department of Transportation - National Highway Traffic Safety Administration, Washington DC, 2008).
    19. 19)
      • 21. Ji, Q., Lan, P., Looney, C.: ‘A probabilistic framework for modeling and real-time monitoring human fatigue’, IEEE Trans. Syst. Man Cybern. A, Syst. Humans, 2006, 36, (5), pp. 862875.
    20. 20)
      • 27. Eskandarian, A., Sayed, R.: ‘Driving simulator experiment: detecting driver fatigue by monitoring eye and steering activity’. Proc. Annual Intelligent Vehicles Systems Symp., Traverse City, Michigan, USA, 2003.
    21. 21)
      • 19. Murphy-Chutorian, E., Doshi, A., Trivedi, M. M.: ‘Head pose estimation for driver assistance systems: a robust algorithm and experimental evaluation’. IEEE Intelligent Transportation Systems Conf., Seattle, Washington, USA, 2007, pp. 709714.
    22. 22)
      • 6. Shiwu, L., Linhong, W., Zhifa, Y., et al: ‘An active driver fatigue identification technique using multiple physiological features’. Int. Conf. on Mechatronic Science, Electric Engineering and Computer (MEC), Jilin, China, 2011, pp. 733737.
    23. 23)
      • 31. Okon, O.D., Meng, L.: ‘Detecting distracted driving with deep learning’. Interactive Collaborative Robotics, Hatfield, United Kingdom, 2017, pp. 170179.
    24. 24)
      • 37. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:14091556, 2014.
    25. 25)
      • 14. Fletcher, L., Zelinsky, A.: ‘Driver state monitoring to mitigate distraction’. Proc. Int. Conf. on the Distractions in Driving, Sydney, Australia, 2007, pp. 487523.
    26. 26)
      • 3. Esurance: ‘3 types of distracted driving’, 2016. Available at:, accessed October 2017.
    27. 27)
      • 43. NVIDIA: ‘Embedded systems’. Available at
    28. 28)
      • 46. Bottou, L.: ‘Large-scale machine learning with stochastic gradient descent’. Proc. COMPSTAT, 2010, pp. 177186.
    29. 29)
      • 18. Kircher, K., Ahlstrom, C., Kircher, A.: ‘Comparison of two eye-gaze based realtime driver distraction detection algorithms in a small-scale field operational test’. Proc. Fifth Int. Symp. on Human Factors in Driver Assessment, Training and Vehicle Design, Big Sky, Montana, USA, 2009, pp. 1623.
    30. 30)
      • 24. Craye, C., Karray, F.: ‘Driver distraction detection and recognition using RGB-D sensor’, CoRR, 2015, abs/1502.00250. Available at:
    31. 31)
      • 30. Colbran, S., Cen, K., Luo, D.: ‘Classification of driver dis- traction’ (Stanford University, Stanford, CA, 2016). Available at:
    32. 32)
      • 17. Pohl, J., Birk, W., Westervall, L.: ‘A driver-distraction-based lane-keeping assistance system’, Proc. Inst. Mech. Eng. I, J. Syst. Control Eng., 2007, 221, (4), pp. 541552.
    33. 33)
      • 11. Farber, E., Foley, J., Scott, S.: ‘Visual attention design limits for its in-vehicle systems: the society of automotive engineers standard for limiting visual distraction while driving’. Transportation Research Board Annual General Meeting, Washington DC, USA, 2000, pp. 23.
    34. 34)
      • 32. Abouelnaga, Y., Eraqi, H.M., Moustafa, M.N.: ‘Real-time distracted driver posture classification’, arXiv preprint arXiv:170609498, 2017.
    35. 35)
      • 38. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘ImageNet classification with deep convolutional neural networks’. Adv. Neural. Inf. Process. Syst., 2012, 1, pp. 10971105.
    36. 36)
      • 35. Venturelli, M., Borghi, G., Vezzani, R., et al: ‘Deep head pose estimation from depth data for in-car automotive applications’, arXiv preprint arXiv:170301883, 2017.
    37. 37)
      • 44. FriendlyARM: ‘Nanopi M3’. Available at
    38. 38)
      • 13. Kutila, M., Jokela, M., Markkula, G., et al: ‘Driver distraction detection with a camera vision system’. IEEE Int. Conf. on Image Processing, San Antonio, Texas, USA, 2007, vol. 6, pp. VI-201VI-204.
    39. 39)
      • 26. Gu, H., Ji, Q.: ‘Facial event classification with task oriented dynamic Bayesian network’. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, CVPR, Washington, DC, USA, 2004, vol. 2, pp. II-870II-875.
    40. 40)
      • 12. Victor, T., Blomberg, O., Zelinsky, A.: ‘Automating the measurement of driver visual behaviours using passive stereo vision’. Proc. Int. Conf. on Series Vision Vehicles (VIV9), Brisbane, Queensland, Australia, 2001.
    41. 41)
      • 39. Szegedy, C., Liu, W., Jia, Y., et al: ‘Going deeper with convolutions’. Proc. IEEE Conf. on computer vision and pattern recognition, Boston, Massachusetts, USA, 2015, pp. 19.
    42. 42)
      • 7. Lal, S.K., Craig, A.: ‘Driver fatigue: electroencephalography and psychological assessment’, Psychophysiology, 2002, 39, (3), pp. 313321.
    43. 43)
      • 22. Craye, C., Karray, F.: ‘Multi-distributions particle filter for eye tracking inside a vehicle’, Image Anal. Recognit., 2013, 6, pp. 407416.
    44. 44)
      • 2. US Department of Transportation – National Highway Traffic Safety Administration: ‘Distracted driving’. Available at:
    45. 45)
      • 34. Choi, I.H., Hong, S.K., Kim, Y.G.: ‘Real-time categorization of driver's gaze zone using the deep learning techniques’. Int. Conf. on Big Data and Smart Computing (BigComp), Jeongseon, Republic of Korea, 2016, pp. 143148.
    46. 46)
      • 36. Hssayeni, M.D., Saxena, S., Ptucha, R., et al: ‘Distracted driver detection: deep learning vs. handcrafted features’, Electron. Imaging, 2017, 7, (10), p. 20–.

Related content

This is a required field
Please enter a valid email address