Driver drowsiness is a frequent cause of traffic accidents. Research on driver drowsiness detection methods is important to improve road traffic safety. Previous driving fatigue detection methods frequently extracted single features such as eye or mouth changes and trained shallow classifiers, which limit the generalisation capability of these methods. This study proposes a framework for recognising driver drowsiness expression by using facial dynamic fusion information and a deep belief network (DBN) to address the aforementioned problem. First, the landmarks and textures of the facial region are extracted from videos captured using a high-definition camera. Then, a DBN is built to classify facial drowsiness expressions. Finally, the authors’ method is tested on a driver drowsiness dataset, which includes different genders, ages, head poses and illuminations. Certain experiments are also carried out to investigate the effects of different facial subregions and temporal resolutions on the accuracy of driver fatigue recognition. Results demonstrate the validity of the proposed method, which has an average accuracy of 96.7%.

References

1. 1)
  - 24. Srivastava, N., Salakhutdinov, R.: ‘Learning representations for multimodal data with deep belief nets’. Proc. Int. Conf. Machine Learning Workshop, Edinburgh, Scotland, UK, July 2012.
2. 2)
  - 15. Oyini Mbouna, R., Kong, S.G., Chun, M.-G.: ‘Visual analysis of eye state and head pose for driver alertness monitoring’, IEEE Trans. Intell. Transp. Syst., 2013, 14, (3), pp. 1462–1469.
3. 3)
  - 26. Kang, S., Qian, X., Meng, H.: ‘Multi-distribution deep belief network for speech synthesis’. Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, May 2013, pp. 8012–8016.
4. 4)
  - 1. Rau, P.S.: ‘Drowsy driver detection and warning system for commercial vehicle drivers: field operational test design, data analyses, and progress’. National Highway Traffic Safety Administration, 2005, pp. 05–0192.
5. 5)
  - 30. Sanin, A., Sanderson, C., Harandi, M.T., et al: ‘Spatio-temporal covariance descriptors for action and gesture recognition’. Proc. IEEE Workshop on Applications of Computer Vision, Tampa, FL, January 2013, pp. 103–110.
6. 6)
  - 28. Asthana, A., Zafeiriou, S., Cheng, S., et al: ‘Incremental face alignment in the wild’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Columbus, OH, USA, June 2014, pp. 1859–1866.
7. 7)
  - 32. Akrout, B., Mahdi, W.: ‘Spatio-temporal features for the automatic control of driver drowsiness state and lack of concentration’, Mach. Vis. Appl., 2015, 26, (1), pp. 1–13.
8. 8)
  - 31. Picot, A., Charbonnier, S., Caplier, A., et al: ‘Using retina modelling to characterize blinking: comparison between EOG and video analysis’, Mach. Vis. Appl., 2012, 23, (6), pp. 1195–1208.
9. 9)
  - 21. Hinton, G.E., Salakhutdinov, R.R.: ‘Reducing the dimensionality of data with neural networks’, Science, 2006, 313, (5786), pp. 504–507.
10. 10)
  - 5. Wang, M., Jeong, N., Kim, K., et al: ‘Drowsy behavior detection based on driving information’, Int. J. Autom. Technol., 2016, 17, (1), pp. 165–173.
11. 11)
  - 16. Gu, H., Ji, Q.: ‘An automated face reader for fatigue detection’. Proc. Sixth IEEE Int. Conf. Automatic Face and Gesture Recognition, Seoul, South Korea, June 2004, pp. 111–116.
12. 12)
  - 22. LeCun, Y., Bottou, L., Bengio, Y., et al: ‘Gradient-based learning applied to document recognition’, Proc. IEEE, 1998, 86, (11), pp. 2278–2324.
13. 13)
  - 17. Fan, X., Sun, Y., Yin, B., et al: ‘Gabor-based dynamic representation for human fatigue monitoring in facial image sequences’, Pattern Recognit. Lett., 2010, 31, (3), pp. 234–243.
14. 14)
  - 4. Kim, J.Y., Jeong, C.H., Jung, M.J., et al: ‘Highly reliable driving workload analysis using driver electroencephalogram (EEG) activities during driving’, Int. J. Autom. Technol., 2013, 14, (6), pp. 965–970.
15. 15)
  - 12. Song, F., Tan, X., Liu, X., et al: ‘Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients’, Pattern Recognit., 2014, 47, (9), pp. 2825–2838.
16. 16)
  - 6. Dinges, D.F., Grace, R.: ‘PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance’. Publication Number FHWA[AQ5] -MCRT-98–006, US Department of Transportation, Federal Highway Administration, 1998.
17. 17)
  - 29. Ji, S., Xu, W., Yang, M., et al: ‘3D convolutional neural networks for human action recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (1), pp. 221–231.
18. 18)
  - 7. Liu, C., Wechsler, H.: ‘Gabor feature based classification using the enhanced Fisher linear discriminant model for face recognition’, IEEE Trans. Image Process., 2002, 11, (4), pp. 467–476.
19. 19)
  - 11. Ibrahim, M.M., Soraghan, J.J., Petropoulakis, L.: ‘Eye-state analysis using an interdependence and adaptive scale mean shift (IASMS) algorithm’, Biomed. Signal Process. Control, 2014, 11, pp. 53–62.
20. 20)
  - 27. Viola, P., Jones, M.J.: ‘Robust real-time face detection’, Int. J. Comput. Vis., 2004, 57, (2), pp. 137–154.
21. 21)
  - 3. Sahayadhas, A., Sundaraj, K., Murugappan, M.: ‘Detecting driver drowsiness based on sensors: a review’, Sensors (Basel), 2012, 12, (12), pp. 16937–16953.
22. 22)
  - 25. Kuremoto, T., Kimura, S., Kobayashi, K., et al: ‘Time series forecasting using a deep belief network with restricted Boltzmann machines’, Neurocomputing, 2014, 137, pp. 47–56.
23. 23)
  - 8. Rong-Ben, W., Ke-You, G., Shu-Ming, S., et al: ‘A monitoring method of driver fatigue behavior based on machine vision’. Proc. Int. Conf. Intelligent Vehicles Symp., Columbus, OH, USA, June 2003, pp. 110–113.
24. 24)
  - 20. Zhao, C., Lian, J., Dang, Q., et al: ‘Classification of driver fatigue expressions by combined curvelet features and Gabor features, and random subspace ensembles of support vector machines’, J. Intell. Fuzzy Syst., 2014, 26, (1), pp. 91–100.
25. 25)
  - 14. Lee, S.J., Jo, J., Jung, H.G., et al: ‘Real-time gaze estimator based on driver's head orientation for forward collision warning system’, IEEE Trans. Intell. Transp. Syst., 2011, 12, (1), pp. 254–267.
26. 26)
  - 23. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, (7553), pp. 436–444.
27. 27)
  - 18. Zhao, C., Zhang, X., Lian, J., et al: ‘Driver's fatigue expressions recognition by combined features from pyramid histogram of oriented gradient and contourlet transform with random subspace ensembles’, IET Intell. Transp. Syst., 2013, 7, (1), pp. 36–45.
28. 28)
  - 13. Cyganek, B., Gruszczyński, S.: ‘Hybrid computer vision system for drivers, eye recognition and fatigue monitoring’, Neurocomputing, 2014, 126, pp. 78–94.
29. 29)
  - 19. Zhang, Y., Hua, C.: ‘Driver fatigue recognition based on facial expression analysis using local binary patterns’, Opt., Int. J. Light Electron Opt., 2015, 126, (23), pp. 4501–4505.
30. 30)
  - 2. Bergasa, L.M., Nuevo, J., Sotelo, M.A., et al: ‘Real-time system for monitoring driver vigilance’, IEEE Trans. Intell. Transp. Syst., 2006, 7, (1), pp. 63–77.
31. 31)
  - 9. Fan, X., Yin, B., Sun, Y.: ‘Nonintrusive driver fatigue detection’. Proc. IEEE Int. Conf. Networking, Sensing and Control, Sanya, China, May 2008, pp. 905–910.
32. 32)
  - 10. Kim, K.W., Lee, W.O., Kim, Y.G., et al: ‘Segmentation method of eye region based on fuzzy logic system for classifying open and closed eyes’, Opt. Eng., 2015, 54, (3), p. 033103.

Driver drowsiness detection using facial dynamic fusion information and a DBN

References

Related content