Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Fatigue driving recognition network: fatigue driving recognition via convolutional neural network and long short-term memory units

  • Author(s): Zhitao Xiao 1, 2 ; Zhiqiang Hu 1, 2 ; Lei Geng 1, 2 ; Fang Zhang 1, 2 ; Jun Wu 1, 2 ; Yuelong Li 2, 3
    • View affiliations
  • Source: Volume 13, Issue 9, September 2019, p. 1410 – 1416
    DOI:  10.1049/iet-its.2018.5392 , Print ISSN 1751-956X, Online ISSN 1751-9578
© The Institution of Engineering and Technology
Received 26/08/2018, Accepted 13/05/2019, Revised 30/04/2019, Published 15/05/2019

Fatigue driving has become one of the major causes of traffic accidents. The authors propose an effective method capable of detecting fatigue state via the spatial–temporal feature of driver's eyes. In this work, the authors consider fatigue detection as image-based sequence recognition and an end-to-end trainable convolutional neural network with long short-term memory (LSTM) units is designed. First, the authors apply a deep cascaded multi-task framework to extract eye region from infrared videos. Then the spatial features are learned by deep convolutional layers and the relationships between adjacent frames are analysed via LSTM units. Finally, through authors’ model, a sequence-level prediction for driving state is produced. The proposed method achieves superior accuracy over the state-of-the-art techniques on authors’ own dataset. Experimental results demonstrate the feasibility of authors’ method.

Inspec keywords: learning (artificial intelligence); neural nets; fatigue; filtering theory; feature extraction

Other keywords: short-term memory units; image-based sequence recognition; spatial–temporal feature; fatigue driving recognition network; traffic accidents; driver; spatial features; fatigue detection; deep cascaded multitask framework; fatigue state; deep convolutional layers; effective method; state-of-the-art techniques; LSTM units; eye region; sequence-level prediction; end-to-end trainable convolutional neural network; authors

Subjects: Knowledge engineering techniques; Other topics in statistics; Neural computing techniques; Filtering methods in signal processing; Optical, image and video signal processing; Computer vision and image processing techniques; Image recognition

References

    1. 1)
      • 20. Hochreiter, S., Schmidhuber, J.: ‘Long short-term memory’, Neural Comput., 1997, 9, (8), pp. 1735–1780.
    2. 2)
      • 25. Ioffe, S., Szegedy, C.: ‘Batch normalization: accelerating deep network training by reducing internal covariate shift’, arXiv preprint arXiv:1502.03167, 2015.
    3. 3)
      • 31. Huang, G., Liu, Z., Maaten, L.V.D., et al: ‘Densely connected convolutional networks’. IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 2261–2269.
    4. 4)
      • 32. Oquab, M., Bottou, L., Laptev, I., et al: ‘Learning and transferring mid-level image representations using convolutional neural networks’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 1717–1724.
    5. 5)
      • 2. Triyanti, V., Iridiastadi, H.: ‘Challenges in detecting drowsiness based on driver's behaviour’, Mater. Sci. Eng. Conf. Series, 2017, 277, (1), p. 012042.
    6. 6)
      • 17. Yann, L.C., Yoshua, B., Geoffrey, H.: ‘Deep learning’, Nature, 2015, 521, (7553), pp. 436–444.
    7. 7)
      • 19. Zhang, F., Su, J., Geng, L., et al: ‘Driver fatigue detection based on eye state recognition’. IEEE Int. Conf. on Machine Vision and Information Technology, Singapore, Singapore, 2017, pp. 105–110.
    8. 8)
      • 5. Fatourechi, M., Bashashati, A., Ward, R.K., et al: ‘EMG and EOG artifacts in brain computer interface systems: A survey’, Clin. Neurophysiol., 2007, 118, (3), pp. 480–494.
    9. 9)
      • 16. Mandal, B., Li, L., Wang, G.S., et al: ‘Towards detection of bus driver fatigue based on robust visual analysis of eye state’, IEEE Trans. Intell. Transp. Syst., 2017, 18, (3), pp. 545–557.
    10. 10)
      • 14. Tao, H., Zhao, Y.: ‘Real-time driver fatigue detection based on face alignment’. Int. Conf. on Digital Image Processing, Hong Kong, 2017, (10420), p. 1042003.
    11. 11)
      • 9. Li, Z., Li, S.E., Li, R., et al: ‘Online detection of driver fatigue using steering wheel angles for real driving conditions’, Sensors, 2017, 17, (3), p. 495.
    12. 12)
      • 26. Hinton, G.E., Srivastava, N., Krizhevsky, A., et al: ‘Improving neural networks by preventing coadaptation of feature detectors’, Comput. Sci., 2012, 3, (4), pp. 212–223.
    13. 13)
      • 18. Ma, S.G., Zhao, C., Sun, H.L, et al: ‘Yawning detection algorithm based on convolutional neural network’, Comput. Sci., 2018, 45, (6A), pp. 227–241.
    14. 14)
      • 29. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘Imagenet classification with deep convolutional neural networks’. Advances in neural information processing systems, Lake Tahoe, NV, USA, 2012, pp. 1097–1105.
    15. 15)
      • 1. Pratama, B.G., Ardiyanto, I., Adji, T.B.: ‘A review on driver drowsiness based on image, bio-signal, and driver behaviour’. IEEE Int. Conf. on Science and Technology-Computer, Yogyakarta, Indonesia, 2017, pp. 70–75.
    16. 16)
      • 21. Hu, J., Shen, L., Sun, G.: ‘Squeeze-and-excitation networks’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, 2018, pp. 7132–7141.
    17. 17)
      • 11. Mittal, A., Kumar, K., Dhamija, S., et al: ‘Head movement-based driver drowsiness detection: A review of state-of-art techniques’. IEEE Int. Conf. on Engineering and Technology, Coimbatore, India, 2016, pp. 903–908.
    18. 18)
      • 3. Mardi, Z., Ashtiani, S.N.M., Mikaili, M.: ‘EEG-based drowsiness detection for safe driving using chaotic features and statistical tests’, J. Med. Signals Sens., 2011, 1, (2), pp. 130–137.
    19. 19)
      • 24. He, K., Zhang, X., Ren, S., et al: ‘Identity mappings in deep residual networks’. European Conf. on computer vision, Amsterdam, Ther Netherlands, 2016, pp. 630–645.
    20. 20)
      • 8. Ma, J., Murphey, Y.L., Zhao, H.: ‘Real time drowsiness detection based on lateral distance using wavelet transform and neural network’. Computational Intelligence. 2015 IEEE Symp., Cape Town, South Africa, 2015, pp. 411–418.
    21. 21)
      • 13. Karchani, M., Mazloumi, A., Saraji, G.N., et al: ‘Presenting a model for dynamic facial expression changes in detecting drivers’ drowsiness’, Electron. Physician., 2015, 7, (2), pp. 1073–1077.
    22. 22)
      • 22. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 770–778.
    23. 23)
      • 4. Jung, S.J., Shin, H.S., Chung, W.Y.: ‘Driver fatigue and drowsiness monitoring system with embedded electrocardiogram sensor on steering wheel’, IET Intell. Transp. Syst., 2014, 8, (1), pp. 43–50.
    24. 24)
      • 30. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, Comput. Sci., 2014, arXiv:1409.1556.
    25. 25)
      • 15. Dinges, D.F., Grace, R.: ‘PERCLOS: a valid psychophysiological measure of alertness as assessed by psychomotor vigilance’. US Department of Transportation, Federal Highway Administration, Publication Number FHWA-MCRT-98-006, 1998.
    26. 26)
      • 6. Gasser, T., Sroka, L., Möcks, J.: ‘The transfer of EOG activity into the EEG for eyes open and closed’, Electroencephalogr. Clin. Neurophysiol., 1985, 61, (2), pp. 181–193.
    27. 27)
      • 23. Zhang, K., Zhang, Z., Li, Z., et al: ‘Joint face detection and alignment using multitask cascaded convolutional networks’, IEEE Signal Process. Lett., 2016, 23, (10), pp. 1499–1503.
    28. 28)
      • 12. Choi, I.H., Kim, Y.G.: ‘Head pose and gaze direction tracking for detecting a drowsy driver’. IEEE Int. Conf. on Big Data and Smart Computing, Bangkok, Thailand, 2014, vol. 9, no. 2, pp. 241–244.
    29. 29)
      • 10. Alioua, N., Amine, A., Rziza, M.: ‘Driver's fatigue detection based on yawning extraction’, Int. J. Vehicular Technol., 2014, 2014, (1), pp. 47–75.
    30. 30)
      • 27. He, K., Zhang, X., Ren, S., et al: ‘Delving deep into rectifiers: surpassing human-level performance on ImageNet classification’. Proc. of the IEEE Int. Conf. on Computer Vision and Pattern Recognition, Boston, MA, USA, 2015, pp. 1026–1034.
    31. 31)
      • 28. Pascanu, R., Mikolov, T., Bengio, Y.: ‘On the difficulty of training recurrent neural networks’. Int. Conf. on Machine Learning, Atlanta, GA USA, 2013, pp. 1310–1318.
    32. 32)
      • 7. Correa, A.G., Orosco, L., Laciar, E.: ‘Automatic detection of drowsiness in EEG records based on multimodal analysis’, Med. Eng. Phys., 2014, 36, (2), pp. 244–249.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2018.5392
Loading

Related content

content/journals/10.1049/iet-its.2018.5392
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address