http://iet.metastore.ingenta.com
1887

Driver drowsiness detection using facial dynamic fusion information and a DBN

Driver drowsiness detection using facial dynamic fusion information and a DBN

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Driver drowsiness is a frequent cause of traffic accidents. Research on driver drowsiness detection methods is important to improve road traffic safety. Previous driving fatigue detection methods frequently extracted single features such as eye or mouth changes and trained shallow classifiers, which limit the generalisation capability of these methods. This study proposes a framework for recognising driver drowsiness expression by using facial dynamic fusion information and a deep belief network (DBN) to address the aforementioned problem. First, the landmarks and textures of the facial region are extracted from videos captured using a high-definition camera. Then, a DBN is built to classify facial drowsiness expressions. Finally, the authors’ method is tested on a driver drowsiness dataset, which includes different genders, ages, head poses and illuminations. Certain experiments are also carried out to investigate the effects of different facial subregions and temporal resolutions on the accuracy of driver fatigue recognition. Results demonstrate the validity of the proposed method, which has an average accuracy of 96.7%.

References

    1. 1)
      • P.S. Rau .
        1. Rau, P.S.: ‘Drowsy driver detection and warning system for commercial vehicle drivers: field operational test design, data analyses, and progress’. National Highway Traffic Safety Administration, 2005, pp. 050192.
        . National Highway Traffic Safety Administration , 05 - 0192
    2. 2)
      • L.M. Bergasa , J. Nuevo , M.A. Sotelo .
        2. Bergasa, L.M., Nuevo, J., Sotelo, M.A., et al: ‘Real-time system for monitoring driver vigilance’, IEEE Trans. Intell. Transp. Syst., 2006, 7, (1), pp. 6377.
        . IEEE Trans. Intell. Transp. Syst. , 1 , 63 - 77
    3. 3)
      • A. Sahayadhas , K. Sundaraj , M. Murugappan .
        3. Sahayadhas, A., Sundaraj, K., Murugappan, M.: ‘Detecting driver drowsiness based on sensors: a review’, Sensors (Basel), 2012, 12, (12), pp. 1693716953.
        . Sensors (Basel) , 12 , 16937 - 16953
    4. 4)
      • J.Y. Kim , C.H. Jeong , M.J. Jung .
        4. Kim, J.Y., Jeong, C.H., Jung, M.J., et al: ‘Highly reliable driving workload analysis using driver electroencephalogram (EEG) activities during driving’, Int. J. Autom. Technol., 2013, 14, (6), pp. 965970.
        . Int. J. Autom. Technol. , 6 , 965 - 970
    5. 5)
      • M. Wang , N. Jeong , K. Kim .
        5. Wang, M., Jeong, N., Kim, K., et al: ‘Drowsy behavior detection based on driving information’, Int. J. Autom. Technol., 2016, 17, (1), pp. 165173.
        . Int. J. Autom. Technol. , 1 , 165 - 173
    6. 6)
      • D.F. Dinges , R. Grace .
        6. Dinges, D.F., Grace, R.: ‘PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance’. Publication Number FHWA[AQ5] -MCRT-98–006, US Department of Transportation, Federal Highway Administration, 1998.
        .
    7. 7)
      • C. Liu , H. Wechsler .
        7. Liu, C., Wechsler, H.: ‘Gabor feature based classification using the enhanced Fisher linear discriminant model for face recognition’, IEEE Trans. Image Process., 2002, 11, (4), pp. 467476.
        . IEEE Trans. Image Process. , 4 , 467 - 476
    8. 8)
      • W. Rong-Ben , G. Ke-You , S. Shu-Ming .
        8. Rong-Ben, W., Ke-You, G., Shu-Ming, S., et al: ‘A monitoring method of driver fatigue behavior based on machine vision’. Proc. Int. Conf. Intelligent Vehicles Symp., Columbus, OH, USA, June 2003, pp. 110113.
        . Proc. Int. Conf. Intelligent Vehicles Symp. , 110 - 113
    9. 9)
      • X. Fan , B. Yin , Y. Sun .
        9. Fan, X., Yin, B., Sun, Y.: ‘Nonintrusive driver fatigue detection’. Proc. IEEE Int. Conf. Networking, Sensing and Control, Sanya, China, May 2008, pp. 905910.
        . Proc. IEEE Int. Conf. Networking, Sensing and Control , 905 - 910
    10. 10)
      • K.W. Kim , W.O. Lee , Y.G. Kim .
        10. Kim, K.W., Lee, W.O., Kim, Y.G., et al: ‘Segmentation method of eye region based on fuzzy logic system for classifying open and closed eyes’, Opt. Eng., 2015, 54, (3), p. 033103.
        . Opt. Eng. , 3 , 033103
    11. 11)
      • M.M. Ibrahim , J.J. Soraghan , L. Petropoulakis .
        11. Ibrahim, M.M., Soraghan, J.J., Petropoulakis, L.: ‘Eye-state analysis using an interdependence and adaptive scale mean shift (IASMS) algorithm’, Biomed. Signal Process. Control, 2014, 11, pp. 5362.
        . Biomed. Signal Process. Control , 53 - 62
    12. 12)
      • F. Song , X. Tan , X. Liu .
        12. Song, F., Tan, X., Liu, X., et al: ‘Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients’, Pattern Recognit., 2014, 47, (9), pp. 28252838.
        . Pattern Recognit. , 9 , 2825 - 2838
    13. 13)
      • B. Cyganek , S. Gruszczyński .
        13. Cyganek, B., Gruszczyński, S.: ‘Hybrid computer vision system for drivers, eye recognition and fatigue monitoring’, Neurocomputing, 2014, 126, pp. 7894.
        . Neurocomputing , 78 - 94
    14. 14)
      • S.J. Lee , J. Jo , H.G. Jung .
        14. Lee, S.J., Jo, J., Jung, H.G., et al: ‘Real-time gaze estimator based on driver's head orientation for forward collision warning system’, IEEE Trans. Intell. Transp. Syst., 2011, 12, (1), pp. 254267.
        . IEEE Trans. Intell. Transp. Syst. , 1 , 254 - 267
    15. 15)
      • R. Oyini Mbouna , S.G. Kong , M.-G. Chun .
        15. Oyini Mbouna, R., Kong, S.G., Chun, M.-G.: ‘Visual analysis of eye state and head pose for driver alertness monitoring’, IEEE Trans. Intell. Transp. Syst., 2013, 14, (3), pp. 14621469.
        . IEEE Trans. Intell. Transp. Syst. , 3 , 1462 - 1469
    16. 16)
      • H. Gu , Q. Ji .
        16. Gu, H., Ji, Q.: ‘An automated face reader for fatigue detection’. Proc. Sixth IEEE Int. Conf. Automatic Face and Gesture Recognition, Seoul, South Korea, June 2004, pp. 111116.
        . Proc. Sixth IEEE Int. Conf. Automatic Face and Gesture Recognition , 111 - 116
    17. 17)
      • X. Fan , Y. Sun , B. Yin .
        17. Fan, X., Sun, Y., Yin, B., et al: ‘Gabor-based dynamic representation for human fatigue monitoring in facial image sequences’, Pattern Recognit. Lett., 2010, 31, (3), pp. 234243.
        . Pattern Recognit. Lett. , 3 , 234 - 243
    18. 18)
      • C. Zhao , X. Zhang , J. Lian .
        18. Zhao, C., Zhang, X., Lian, J., et al: ‘Driver's fatigue expressions recognition by combined features from pyramid histogram of oriented gradient and contourlet transform with random subspace ensembles’, IET Intell. Transp. Syst., 2013, 7, (1), pp. 3645.
        . IET Intell. Transp. Syst. , 1 , 36 - 45
    19. 19)
      • Y. Zhang , C. Hua .
        19. Zhang, Y., Hua, C.: ‘Driver fatigue recognition based on facial expression analysis using local binary patterns’, Opt., Int. J. Light Electron Opt., 2015, 126, (23), pp. 45014505.
        . Opt., Int. J. Light Electron Opt. , 23 , 4501 - 4505
    20. 20)
      • C. Zhao , J. Lian , Q. Dang .
        20. Zhao, C., Lian, J., Dang, Q., et al: ‘Classification of driver fatigue expressions by combined curvelet features and Gabor features, and random subspace ensembles of support vector machines’, J. Intell. Fuzzy Syst., 2014, 26, (1), pp. 91100.
        . J. Intell. Fuzzy Syst. , 1 , 91 - 100
    21. 21)
      • G.E. Hinton , R.R. Salakhutdinov .
        21. Hinton, G.E., Salakhutdinov, R.R.: ‘Reducing the dimensionality of data with neural networks’, Science, 2006, 313, (5786), pp. 504507.
        . Science , 5786 , 504 - 507
    22. 22)
      • Y. LeCun , L. Bottou , Y. Bengio .
        22. LeCun, Y., Bottou, L., Bengio, Y., et al: ‘Gradient-based learning applied to document recognition’, Proc. IEEE, 1998, 86, (11), pp. 22782324.
        . Proc. IEEE , 11 , 2278 - 2324
    23. 23)
      • Y. LeCun , Y. Bengio , G. Hinton .
        23. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, (7553), pp. 436444.
        . Nature , 7553 , 436 - 444
    24. 24)
      • N. Srivastava , R. Salakhutdinov .
        24. Srivastava, N., Salakhutdinov, R.: ‘Learning representations for multimodal data with deep belief nets’. Proc. Int. Conf. Machine Learning Workshop, Edinburgh, Scotland, UK, July 2012.
        . Proc. Int. Conf. Machine Learning Workshop
    25. 25)
      • T. Kuremoto , S. Kimura , K. Kobayashi .
        25. Kuremoto, T., Kimura, S., Kobayashi, K., et al: ‘Time series forecasting using a deep belief network with restricted Boltzmann machines’, Neurocomputing, 2014, 137, pp. 4756.
        . Neurocomputing , 47 - 56
    26. 26)
      • S. Kang , X. Qian , H. Meng .
        26. Kang, S., Qian, X., Meng, H.: ‘Multi-distribution deep belief network for speech synthesis’. Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Vancouver, BC, Canada, May 2013, pp. 80128016.
        . Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing , 8012 - 8016
    27. 27)
      • P. Viola , M.J. Jones .
        27. Viola, P., Jones, M.J.: ‘Robust real-time face detection’, Int. J. Comput. Vis., 2004, 57, (2), pp. 137154.
        . Int. J. Comput. Vis. , 2 , 137 - 154
    28. 28)
      • A. Asthana , S. Zafeiriou , S. Cheng .
        28. Asthana, A., Zafeiriou, S., Cheng, S., et al: ‘Incremental face alignment in the wild’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Columbus, OH, USA, June 2014, pp. 18591866.
        . Proc. IEEE Conf. Computer Vision and Pattern Recognition , 1859 - 1866
    29. 29)
      • S. Ji , W. Xu , M. Yang .
        29. Ji, S., Xu, W., Yang, M., et al: ‘3D convolutional neural networks for human action recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (1), pp. 221231.
        . IEEE Trans. Pattern Anal. Mach. Intell. , 1 , 221 - 231
    30. 30)
      • A. Sanin , C. Sanderson , M.T. Harandi .
        30. Sanin, A., Sanderson, C., Harandi, M.T., et al: ‘Spatio-temporal covariance descriptors for action and gesture recognition’. Proc. IEEE Workshop on Applications of Computer Vision, Tampa, FL, January 2013, pp. 103110.
        . Proc. IEEE Workshop on Applications of Computer Vision , 103 - 110
    31. 31)
      • A. Picot , S. Charbonnier , A. Caplier .
        31. Picot, A., Charbonnier, S., Caplier, A., et al: ‘Using retina modelling to characterize blinking: comparison between EOG and video analysis’, Mach. Vis. Appl., 2012, 23, (6), pp. 11951208.
        . Mach. Vis. Appl. , 6 , 1195 - 1208
    32. 32)
      • B. Akrout , W. Mahdi .
        32. Akrout, B., Mahdi, W.: ‘Spatio-temporal features for the automatic control of driver drowsiness state and lack of concentration’, Mach. Vis. Appl., 2015, 26, (1), pp. 113.
        . Mach. Vis. Appl. , 1 , 1 - 13
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2017.0183
Loading

Related content

content/journals/10.1049/iet-its.2017.0183
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address