access icon openaccess Driver emotion recognition of multiple-ECG feature fusion based on BP network and D–S evidence

This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/)
Received 27/07/2019, Accepted 18/02/2020, Revised 10/12/2019, Published 02/03/2020

Driving emotion is considered as driver's psychological reaction to a change in traffic environment, which affects driver's cognitive, judgement and behaviour. In anxiety, drivers are more likely to get engaged in distracted driving, increasing the likelihood of vehicle crash. Therefore, it is essential to identify driver's anxiety during driving, to provide a basis for driving safety. This study used multiple-electrocardiogram (ECG) feature fusion to recognise driver's emotion, based on back-propagation network and Dempster–Shafer evidence method. The three features of ECG signals, the time–frequency domain, waveform and non-linear characteristics were selected as the parameters for emotion recognition. An emotion recognition model was proposed to identify drivers’ calm and anxiety during driving. The results show after ECG evidence fusion, the proposed model can recognise drivers’ emotion, with an accuracy rate of 91.34% for calm and 92.89% for anxiety. The authors’ findings of this study can be used to develop the personalised driving warning system and intelligent human–machine interaction in vehicles. This study would be of great theoretical significance and application value for improving road traffic safety.

Inspec keywords: backpropagation; cognition; road accidents; road traffic; driver information systems; uncertainty handling; electrocardiography; sensor fusion; emotion recognition; road safety; inference mechanisms; psychology; feature extraction; medical signal processing

Other keywords: multiple-ECG feature fusion; ECG signals; Dempster–Shafer evidence method; distracted driving; emotion recognition model; driver emotion recognition; nonlinear characteristics; road traffic safety; driver psychological reaction; BP network; traffic environment; time–frequency domain; D–S evidence; ECG evidence fusion; personalised driving warning system; driving emotion; anxiety; multiple-electrocardiogram feature fusion; back-propagation network; driving safety; vehicle crash

Subjects: Neural computing techniques; Biology and medical computing; Electrical activity in neurophysiological processes; Traffic engineering computing; Bioelectric signals; Signal processing and detection; Electrodiagnostics and other electrical measurement techniques; Knowledge engineering techniques; Sensor fusion

References

    1. 1)
      • 11. Wang, X., Zhang, J.L., Ban, X., et al: ‘Study on transformation mechanism of driver's propensity under two-lane conditions', Appl. Mech. Mater., 2013, 34, (9), pp. 411–414.
    2. 2)
      • 4. Wang, X., Liu, Y., Guo, Y., et al: ‘Transformation mechanism of vehicle cluster situations under dynamic evolution of driver's propensity’, Transp. Res. F, Psychol. Behav., 2018, 12, (7), pp. 21–29.
    3. 3)
      • 20. Wan, P., Wu, C., Lin, Y., et al: ‘Driver's angry emotion recognition model based on confidence rule base’, J. Transp. Syst. Eng. Inf., 2015, 15, (5), pp. 96–102.
    4. 4)
      • 28. Bradley, M., Lang, P.: ‘International affective picture system’, in Zeigler-Hill, V., Shackelford, T. (Eds.): ‘Encyclopedia of personality and individual differences’ (Springer International Publishing, Berlin, Germany, 2017), pp. 1–4.
    5. 5)
      • 19. Minhad, K., Md Ali, S.: ‘Happy-anger emotions classifications from electrocardiogram signal for automobile driving safety and awareness’, J. Transp. Health., 2017, 7, pp. 75–89.
    6. 6)
      • 26. Ling, M., Wang, J., Wang, X.: ‘Multi-features fusion diagnosis of tremor based on artificial neural network and D–S evidence theory’, Signal Process., 2008, 88, (12), pp. 2927–2935.
    7. 7)
      • 1. Wang, X, Zhang, J, Ban, X.: ‘Identification of automobile driving tendency based on collaborative simulation of dynamic human-car environment’, Sci. Press Beijing, 2013, pp. 1–3.
    8. 8)
      • 9. Wang, X., Liu, Y., Guo, Y., et al: ‘Transformation mechanism of vehicle cluster situations under dynamic evolution of driver's propensity’, Transp. Res. F, Psychol. Behav., 2017, 234, (2), pp. 117–129.
    9. 9)
      • 21. Guo, X.: ‘Study of emotion recognition based on electrocardiogram and RBF neural network’, Procedia Eng., 2011, 15, pp. 2408–2412.
    10. 10)
      • 15. Wang, X., Zhang, J.: ‘Extraction and recognition methods of vehicle driving tendency feature based on driver-vehicle-environment dynamic data under car following’, Int. J. Comput. Intell. Syst., 2011, 4, (6), pp. 1269–1281.
    11. 11)
      • 30. Guo, Y., Wang, X., Xu, Q., et al: ‘Change-point analysis of eye movement characteristics for female drivers in anxiety’, Int. J. Environ. Res. Public Health, 2019, 16, (7), p. 1236.
    12. 12)
      • 10. Wang, X., Wang, J., Zhang, J., et al: ‘Dynamic recognition of driver's propensity based on GPS mobile sensing data and privacy protection’, Math. Probl. Eng., 2016, 14, (8), pp. 18–29. Article ID 1814608.2016.
    13. 13)
      • 27. Bradley, M., Lang, P.: ‘The International Affective Picture System (IAPS) in the study of emotion and attention’, in Coan, J.A., Allen, J.J.B. (Eds.): ‘Series in affective science. Handbook of emotion elicitation and assessment’ (Oxford University Press, Oxford, UK, 2007), pp. 29–46.
    14. 14)
      • 29. Bai, L., Ma, H., Huang, Y., et al: ‘The development of native Chinese affective picture system – a pretest in 46 college students’, Chin. Mental Health J., 2005, 19, (11), pp. 719–722.
    15. 15)
      • 8. Geethanjali, B., Adalarasu, K., Jagannath, M., et al: ‘Music induced emotion using wavelet packet decomposition – an EEG study’, Biomed. Signal Process. Control, 2018, 45, pp. 115–128.
    16. 16)
      • 17. Scott-Parker, B.: ‘Emotions, behavior, and the adolescent driver: a literature review’, Transp. Res. F, Traffic Psychol. Behav., 2017, 50, pp. 1–37.
    17. 17)
      • 13. Wang, X., Zhang, J., Ban, X., et al: ‘Dynamic feature extraction method of driver's propensity under complicated vehicle group’, Adv. Mech. Eng., 2013, 287, (4), pp. 140–150. Article ID 287653, 10 pages.2013.
    18. 18)
      • 14. Wang, X., Liu, J., Zhang, J.: ‘Dynamic recognition model of driver's propensity under multilane traffic environments’, Discret. Dyn. Nat. Soc., 2015, 15, (10), pp. 54–69.
    19. 19)
      • 24. Guo, K., Li, W.: ‘Combination rule of D–S evidence theory based on the strategy of cross-merging between evidence’, Expert Syst. Appl., 2011, 38, (10), pp. 13360–13366.
    20. 20)
      • 23. Hyun-Min, L.: ‘A modified Zeeman model for producing HRV signals and its application to ECG signal generation’, J. Theor. Biol., 2007, 244, (2), pp. 180–189.
    21. 21)
      • 25. Qu, X., Yu, W., Zhao, Y.: ‘Fatigue driving dangerous state identification based on entropy weight grey relation and D–S evidence theory’, J. Automot. Saf. Energy Conserv., 2018, 9, (2), pp. 164–170.
    22. 22)
      • 6. Schmidt-Daffy, M.: ‘Velocity versus safety: impact of goal conflict and task difficulty on drivers’ behavior, feeling of anxiety, and electro-dermal responses’, Transp. Res. F, Traffic Psychol. Behav., 2012, 15, pp. 319–332.
    23. 23)
      • 3. Wang, X., Liu, Y., Wang, J., et al: ‘Study on influencing factors selection of driver's propensity’, Transp. Res. D, Transp. Environ., 2018, 15, (4), pp. 17–25.
    24. 24)
      • 5. Wang, X., Wang, J., Zhang, J., et al: ‘Driver's behavior and decision-making optimization model in mixed traffic environment international’, Adv. Mech. Eng., 2015, 14, (9), pp. 32–39.
    25. 25)
      • 22. Yu, C., Wang, C., Zheng, J., et al: ‘Physiological emotion analysis using support vector regression’, Neurocomputing, 2013, 122, (25), pp. 79–87.
    26. 26)
      • 18. Kessous, L., Castellano, G., Carisakis, G.: ‘Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis’, J. Multimodal User Interfaces, 2010, 3, (1), pp. 33–48.
    27. 27)
      • 12. Wang, X., Wang, J., Liu, Y., et al: ‘Reverse deduction of vehicle group situation based on dynamic Bayesian network’, Adv. Mech. Eng., 2018, 10, (3), pp. 1–15.
    28. 28)
      • 16. Wan, P., Wu, C., Lin, Y., et al: ‘Monitoring of angry driving status based on multiple time-series characteristics of driving behavior’, J. Jilin Univ. Eng. Sci., 2017, 47, (5), pp. 1426–1435.
    29. 29)
      • 2. Jia, H., Si, Y., Tang, M.: ‘Research on driver information processing model based on cognitive psychology’, Chin. J. Saf. Sci., 2006, 16, (1), pp. 22–25.
    30. 30)
      • 7. Wang, X., Wang, J., Zhang, J., et al: ‘Driver's behavior and decision-making optimization model in mixed traffic environment’, Int. Adv. Mech. Eng., 2015, 17, (3), pp. 27–38.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-its.2019.0499
Loading

Related content

content/journals/10.1049/iet-its.2019.0499
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading