Smartphone-based fatigue detection system using progressive locating method

Zibo Li; Guangmin Sun; Fan Zhang; Linan Jia; Kun Zheng; Dequn Zhao

Smartphone-based fatigue detection system using progressive locating method

View Fulltext

Author(s): Zibo Li ¹ ; Guangmin Sun ^{1, 2} ; Fan Zhang ¹ ; Linan Jia ¹ ; Kun Zheng ¹ ; Dequn Zhao ¹
- Affiliations: 1: Department of Electronic Engineering, Beijing University of Technology, Beijing 100124, People's Republic of China ;
  2: Beijing Key Laboratory of Computational Intelligence and Intelligent System, Beijing University of Technology, Beijing 100124, People's Republic of China
Source: Volume 10, Issue 3, April 2016, p. 148 – 156
DOI: 10.1049/iet-its.2015.0076 , Print ISSN 1751-956X, Online ISSN 1751-9578

Received 17/04/2015, Accepted 18/09/2015, Revised 24/07/2015, Published 01/04/2016

Smartphone applications became very popular nowadays as they provide useful functionalities to our daily lives over ordinary voice services. They offer a small but powerful computing platform where intelligent algorithms can be coded into some live-saving products which profoundly impact our daily lifestyles. A mobile fatigue detection system which is proposed in this study is an important life-saving application running on smartphone. Mobile detection system faces technological challenges such as: (i) embracing relatively low-resolution images in image recognition; (ii) supporting fast response time given a low-power CPU in comparison to a desktop computer; and (iii) demanding for high prediction accuracy by light-weight machine learning algorithms, as the software programs embedded in smartphones is resource constrained. Solutions with respect to indoor facial profiling system which are mainly based on progressive locating method for eye detection are discussed in this study. Acceptable experimental results in terms of eye detection rate and driver fatigue detection in different situations are presented.

References

1. 1)
  - 17. Luo, X., Hu, R., Fan, T.: ‘The driver fatigue monitoring system based on face recognition technology’. 2013 Fourth Int. Conf. on Intelligent Control and Information Processing, Beijing, China, 9–11 June 2013, pp. 384–388.
2. 2)
  - 9. Ibrahim, L.F., Abulkhair, M., AlShomrani, A.D., et al: ‘Using Haar classifiers to detect driver fatigue and provide alerts’, Multimedia Tools Appl.71, (3), pp. 1857–1877 (doi: 10.1007/s11042-012-1308-5).
3. 3)
  - 18. Xiong, C., Xie, M., Wang, L.: ‘Driver fatigue detection based on AdaBoost’, Adv. Intell. Soft Comput., 2012, 159, pp. 13–17 (doi: 10.1007/978-3-642-29387-0_3).
4. 4)
  - 23. Xinjin, Wei.: ‘The study of fatigue detection of drivers based on the infrared conditions’. Master thesis, Zhejiang Sci-Tech University, 2011.
5. 5)
  - 11. Lala, S.K.L., Craiga, A., Boorda, P., et al: ‘Development of an algorithm for an EEG-based driver fatigue countermeasure’, J. Saf. Res., 2003, 34, pp. 321–328 (doi: 10.1016/S0022-4375(03)00027-6).
6. 6)
  - 15. Cyganek, B., Gruszczyński, S.: ‘Hybrid computer vision system for drivers’ eye recognition and fatigue monitoring’, Neurocomputing, 2014, 126, pp. 78–94 (doi: 10.1016/j.neucom.2013.01.048).
7. 7)
  - 2. Thiffault, P., Bergeron, J.: ‘Monotony of road environment and driver fatigue: a simulator study’, Accident Anal. Prev., 2003, 35, pp. 381–391 (doi: 10.1016/S0001-4575(02)00014-3).
8. 8)
  - 38. Fan, X., Yin, B.-c., Sun, Y.-f.: ‘Yawning detection for monitoring driver fatigue’. Proc. Sixth Int. Conf. on Machine Learning and Cybernetics, Hong Kong, 19–22 August 2007, pp. 664–668.
9. 9)
  - 32. Wang, Q., Yang, J.Y.: ‘Eye location and eye state detection in facial images with unconstrained background’, J. Inf. Comput. Sci., 2006, 1, (5), pp. 284–289.
10. 10)
  - 8. Eskandarian, A., Mortazavi, A.: ‘Evaluation of a smart algorithm for commercial vehicle driver drowsiness detection’. IEEE Intelligent Vehicles Symp., Istanbul, Turkey, 13–15 June 2007, pp. 553–559.
11. 11)
  - 30. Alioua, N., Amine, A., Rziza, M., et al: ‘Driver's fatigue and drowsiness detection to reduce traffic accidents on road’. 14th Int. Conf. on Computer Analysis of Images and Patterns, Seville, Spain, August 2011, pp. 397–404.
12. 12)
  - 12. Patel, M., Lal, S.K.L., Kavanagh, D., et al: ‘Applying neural network analysis on heart rate variability data to assess driver fatigue’, Expert Syst. Appl., 2011, 38, pp. 7235–7242 (doi: 10.1016/j.eswa.2010.12.028).
13. 13)
  - 31. Soylemez, O.F., Ergen, B.: ‘Eye location and eye state detection in facial images using circular Hough transform’. 12th IFIP TC8 Int. Conf., Krakow on Computer Information Systems and Industrial Management, Poland, September 2013, pp. 141–147.
14. 14)
  - 6. Ingre, M., Kerstedt, T., Peters, , et al: ‘Subjective sleepiness and accident risk avoiding the ecological fallacy’, J. Sleep Res., 2006, 15, pp. 142–148 (doi: 10.1111/j.1365-2869.2006.00517.x).
15. 15)
  - 4. Merat, N., Hamish Jamson, A.: ‘The effect of three low-cost engineering treatments on driver fatigue: a driving simulator study’, Accident Anal. Prev., 2013, 50, pp. 8–15 (doi: 10.1016/j.aap.2012.09.017).
16. 16)
  - 13. Zhang, L., Liu, F., Tang, J.: ‘Real-time system for driver fatigue detection by RGB-D camera’, ACM Trans. Intell. Syst. Technol., 2015, 6, (2), pp. 1–17.
17. 17)
  - 14. Azim, T., Arfan Jaffar, M., Mirzada, A.M.: ‘Fully automated real time fatigue detection of drivers through fuzzy expert systems’, Appl. Soft Comput., 2014, 18, pp. 25–38 (doi: 10.1016/j.asoc.2014.01.020).
18. 18)
  - 20. Phillips, J., Wechsler, H., Huang, J.S., et al: ‘The FERET database and evaluation procedure for face recognition algorithms’, Image Vis. Comput., 1998, pp. 295–306 (doi: 10.1016/S0262-8856(97)00070-X).
19. 19)
  - 19. Yong, D., Degang, C., Qinghua, H., et al: ‘Kernelized fuzzy rough sets based yawn detection for driver fatigue monitoring’, Fundam. Inform., 111, (1), pp. 65–79.
20. 20)
  - 33. Yuille, A., Hallinan, P., Cohen, D., et al: ‘Feature extraction from faces using deformable templates’, Int. J. Comput. Vis., 1992, 8, (2), pp. 99–111 (doi: 10.1007/BF00127169).
21. 21)
  - 22. Ren, Y., Wang, S., Hou, B., et al: ‘A novel eye localization method with rotation invariance’, IEEE Trans. Image Process., 2014, 23, (1), pp. 226–239 (doi: 10.1109/TIP.2013.2287614).
22. 22)
  - 16. McCall, J.C., Trivedi, M.M.: ‘Visual context capture and analysis for driver attention monitoring’. 2004 IEEE Intelligent Transprtation Systems Conf., Washington, DC, USA, 3–6 October 2004, pp. 332–337.
23. 23)
  - 10. He, Q., Li, W., Fan, X., et al: ‘Driver fatigue evaluation model with integration of multi-indicators based on dynamic Bayesian network’, IET Intell. Transp. Syst., 2015, 9, (5), pp. 547–554 (doi: 10.1049/iet-its.2014.0103).
24. 24)
  - 1. Ahmed, R., Emon, K.E.K., Foisal Hossain, Md.: ‘Robust driver fatigue recognition using image processing’. Third Int. Conf. on Informatics, Electronics, and Vision, Dhaka, The People's Republic of Bangladesh, May 2014, pp. 1–6.
25. 25)
  - 36. Wierwille, W., Ellsworth, L., Wreggit, S., et al: ‘Research on vehicle-based driver status/performance monitoring: development, validation and refinement of algorithms for detection of driver drowsiness’. DOT HS 808 247, National Highway Traffic Safety Administration, 1994,.
26. 26)
  - 29. Zhang, Z., Zhang, J.: ‘A new real-time eye tracking based on nonlinear unscented Kalman filter for monitoring driver fatigue’, J. Control Theory Appl., 2010, 8, (2), pp. 181–188 (doi: 10.1007/s11768-010-8043-0).
27. 27)
  - 37. Phillips, P., Moon, H., Rizvi, S., Rauss, P.: ‘The FERET evaluation methodology for face-recognition algorithms’, IEEE Trans. Pattern Anal. Mach. Intell., 2000, 22, (10), pp. 1090–1104 (doi: 10.1109/34.879790).
28. 28)
  - 20. Buscarino, A., Fortuna, L., Frasca, M.: ‘Driving assistance using smartdevices’. 2014 IEEE Int. Symp. on Intelligent Control (ISIC), Part of 2014 IEEE Multi-Conf. on Systems and Control, Antibes, France, 8–10 October 2014, pp. 838–842.
29. 29)
  - 5. Boyle, L., Tippin, J., Paul, A., et al: ‘Driver performance declines in the moments surrounding a microsleep’, Transp. Res. F, Traffic Psychol. Behav., 2008, 11, pp. 126–136 (doi: 10.1016/j.trf.2007.08.001).
30. 30)
  - 27. Gercia, H., Salazar, A., Alvarez, D., et al: ‘Driving fatigue detection using active shape models’. Sixth Int. Symp. on Advances in Visual Computing, Las Vegas, NV, USA, November 2010, pp. 171–180.
31. 31)
  - 26. Everingham, M., Zisserman, A.: ‘Regression and classification approaches to eye localization in face images’. Seventh Int. Conf. on Automatic Face Gesture Recognition, Southampton, UK, April 2006, pp. 2105–2112.
32. 32)
  - 39. Azim, T., Jaffar, A., Mirza, A., et al: ‘Automatic fatigue detection of drivers through pupil detection and yawning analysis’. 2009 Fourth Int. Conf. on Innovative Computing, Information and Control, Kaohsiung, Taiwan, 7–9 December 2009, pp. 441–445.
33. 33)
  - 7. Syed, R., Eskandarian, A.: ‘Unobtrusive drowsiness detection by neural network learning of driver steering’, Proc. Inst. Mech. Eng. D, J. Automob. Eng., 2001, 215, (9), pp. 969–975 (doi: 10.1243/0954407011528536).
34. 34)
  - 25. Li, S.Z., Chu, R.F., Liao, S.C., et al: ‘Illumination invariant face recognition using near-infrared images’, IEEE Trans. Pattern Anal. Mach. Intell., 2007, 29, (4), pp. 627–639 (doi: 10.1109/TPAMI.2007.1014).
35. 35)
  - 21. He, J., Roberson, S., Fields, B., et al: ‘Fatigue detection using smartphones’, J. Ergon., 2013, 3, pp. 1–7.
36. 36)
  - 28. Valenti, R., Gevers, T.: ‘Accurate eye center location through invariant isocentric pattern’, IEEE Trans. Pattern Anal. Mach. Learn., 2012, 34, (8), pp. 1785–1798 (doi: 10.1109/TPAMI.2011.251).
37. 37)
  - 24. Reinders, M.J.T., Koch, R.W.C., Gerbrands, J.J.: ‘Locating facial features in image sequences using neural networks automatic face and gesture recognition’. Proc. Second Int. Conf. on Automatic Face and Gesture Recognition, Killington, USA, October 1996, pp. 230–235.
38. 38)
  - 3. Karrer, K., Roetting, M.: ‘Effects of driver fatigue monitoring – an expert survey’. Seventh Int. Conf. on Engineering Psychology and Cognitive Ergonomics, Beijing, China, 22–27 July 2007, pp. 324–330.
39. 39)
  - 37. Li, L., Chen, Y., Li, Z.: ‘Yawning detection for monitoring driver fatigue based on two cameras’. Proc. 12th Int. IEEE Conf. on Intelligent Transportation Systems, St. Louis, USA, 3–7 October 2009, pp. 12–17.

Login

Not registered yet?

Share

Tools

Login to add to favourites

Key

Smartphone-based fatigue detection system using progressive locating method

References

Related content