© The Institution of Engineering and Technology
A real-time thermal imaging based, non-contact respiration rate monitoring method was developed. It measured the respiration related skin surface temperature changes under the tip of the nose. Facial tracking was required as head movements caused the face to appear in different locations in the recorded images over time. The algorithm detected the tip of the nose and then, a region just under it was selected. The pixel values in this region in successive images were processed to determine respiration rate. The segmentation method, used as part of the facial tracking, was evaluated on 55,000 thermal images recorded from 14 subjects with different extent of head movements. It separated the face from image background in all images. However, in 11.7% of the images, a section of the neck was also included, but this did not cause an error in determining respiration rate. The method was further evaluated on 15 adults, against two contact respiration rate monitoring methods that tracked thoracic and abdominal movements. The three methods gave close respiration rates in 12 subjects but in 3 subjects, where there were very large head movements, the respiration rates did not match.
References
-
-
1)
-
40. Farkas, L.G., Katic, M.J., Forrest, C.R., et al: ‘International anthropometric study of facial morphology in various ethnic groups/races’, J. Craniofac. Surg., 2005, 16, (4), pp. 615–646 (doi: 10.1097/01.scs.0000171847.58031.9e).
-
2)
-
22. Viola, P., Jones, M.: ‘Rapid object detection using a boosted cascade of simple features’. Proc. Conf. on Computer and Pattern Recognition, 2001. .
-
3)
-
32. Shastri, D., Pavlidis, I.: ‘Automatic initiation of the periorbital signal extraction in thermal imagery’. Advanced Video and Signal Based Surveillance, IEEE Computer Society, IEE Explore, 2009, pp. 182–186. .
-
4)
-
20. Yang, M.H., Kriegman, D., Ahuja, N.: ‘Detecting faces in images: a survey’, IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24, (1), pp. 34–58 (doi: 10.1109/34.982883).
-
5)
-
7. AL-Khalidi, F.Q., Saatchi, R., Burke, D., et al: ‘Respiration rate monitoring methods: a review’, Pediatr. Pulmonol., 2011, 46, (6), pp. 523–529 (doi: 10.1002/ppul.21416).
-
6)
-
3. Ambrogio, C., Parthasarathy, S.: ‘Polysomnography during critical illness’, J. Clin. Sleep Med., 2007, 3, (6), pp. 649–650.
-
7)
-
19. Yang, G., Huang, T.S.: ‘Human face detection in a complex background’, Pattern Recognit., 1994, 27, (1), pp. 53–63 (doi: 10.1016/0031-3203(94)90017-5).
-
8)
-
11. Min, S.D., Kim, J.K., Shin, H.S., et al: ‘Noncontact respiration rate measurement system using an ultrasonic proximity sensor’, IEEE Sens. J., 2010, 10, pp. 1732–1739 (doi: 10.1109/JSEN.2010.2044239).
-
9)
-
10. Min, S.D., Yoon, D.J., Yoon, S.W., et al: ‘A study on a non-contacting respiration signal monitoring system using Doppler ultrasound’, Med. Biol. Eng. Comput., 2007, 45, (11), pp. 1113–1119 (doi: 10.1007/s11517-007-0246-2).
-
10)
-
33. Zhou, Y., Tsiamyrtzis, P., Lindner, P., et al: ‘Spatiotemporal smoothing as a basis for facial tissue tracking in thermal imaging’, IEEE Trans. Biomed. Eng., 2013, 60, (5), pp. 1280–1289 (doi: 10.1109/TBME.2012.2232927).
-
11)
-
34. Edelman, G.J., Hoveling, R.J., Roos, M., et al: ‘Infrared imaging of the crime scene: possibilities and pitfalls’, J. Forensic Sci., 2013, 58, (5), pp. 1156–1162 (doi: 10.1111/1556-4029.12225).
-
12)
-
24. Hanawa, D., Morimoto, T., Shimazaki, S., et al: ‘Nasal cavity detection in facial thermal image for non-contact measurement of breathing’, Int. J. Adv. Telecommun. Electrotechnics Signals Syst., 2013, 2, (1), pp. 33–38.
-
13)
-
15. Otsu, N.: ‘A threshold selection method from gray-level histograms’, IEEE Trans. Syst. Man Cybern., 1979, 9, (1), pp. 62–66 (doi: 10.1109/TSMC.1979.4310076).
-
14)
-
5. Sampol, G., Romero, O., Salas, A., et al: ‘Obstructive sleep apnea and thoracic aorta dissection’, Am. J. Respir. Crit. Care Med., 2003, 168, (12), pp. 1528–1531 (doi: 10.1164/rccm.200304-566OC).
-
15)
-
21. Mostafa, E., Farag, A., Shalaby, A., et al: ‘Long term facial parts tracking in thermal imaging for uncooperative emotion recognition’. IEEE Xplore, 2013. .
-
16)
-
9. Poh, M.Z., McDuff, D.J., Picard, R.W.: ‘Advancements in noncontact, multiparameter physiological measurements using a webcam’, IEEE Trans. Biomed. Eng., 2011, 58, (1), pp. 7–11 (doi: 10.1109/TBME.2010.2086456).
-
17)
-
13. Aoki, H., Takemura, Y., Mimura, K., et al: ‘Development of non-restrictive sensing system for sleeping person using fiber grating vision sensor’. 2001 Int. Symp. on Micromechatronics and Human Science, 2001, pp. 155–160.
-
18)
-
9. Greneker, E.F.: ‘Radar sensing of heartbeat and respiration at a distance with applications of the technology’. Radar 97 (Conf. Publ. no. 449), 1997, pp. 150–154.
-
19)
-
30. Murthy, J.N., van Jaarsveld, J., Fei, J., et al: ‘Thermal infraed imaging: a novel method to monitor airflow during polysomnography’, Sleep, 2009, 32, (11), pp. 1521–1527.
-
20)
-
18. Jones, B.F., Plassmann, P.: ‘Digital infrared thermal imaging of human skin’, IEEE Eng. Med. Biol., 2002, 21, pp. 41–48 (doi: 10.1109/MEMB.2002.1175137).
-
21)
-
4. Parthasarathy, S., Tobin, M.J.: ‘Sleep in the intensive care unit’, Intensive Care Med., 2004, 30, (2), pp. 197–206 (doi: 10.1007/s00134-003-2030-6).
-
22)
-
27. Fei, J., Zhu, Z., Pavlidis, I.: ‘Imaging breathing rate in the CO2 absorption band’. Proc. 27th Annual Conf of the IEEE Engineering in Medicine and Biology, 2005, pp. 700–705.
-
23)
-
37. Tsai, W.: ‘Moment-preserving thresolding: a new approach’, Comput. Vis. Graph. Image Process., 1985, 29, pp. 377–393. (doi: 10.1016/0734-189X(85)90133-1).
-
24)
-
29. Fei, J., Pavlidis, I.: ‘Thermistor at a distance: unobtrusive measurement of breathing’, IEEE Trans. Biomed. Eng., 2010, 57, (4), pp. 988–998 (doi: 10.1109/TBME.2009.2032415).
-
25)
-
25. AL-Khalidi, F.Q., Saatchi, R., Burke, D., et al: ‘Tracking human face features in thermal images for respiration monitoring’. 2010 IEEE/ACS Int. Conf. on Computer Systems and Applications, 2010, pp. 1–6.
-
26)
-
17. Alkali, A.H., Saatchi, R., Elphick, H., et al: ‘Noncontact respiration rate monitoring based on sensing exhaled air’, Malaysian J. Fundam. Appl. Sci., 2013, 9, (3), pp. 129–133.
-
27)
-
26. Martinez, B., Binefa, X., Pantic, M.: ‘Facial component detection in thermal imagery’. 2010 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2010, pp. 48–54.
-
28)
-
8. Bartula, M., Tigges, T., Muehlsteff, J.: ‘Camera-based system for contactless monitoring of respiration’. Conf. Proc. IEEE Engineering in Medicine and Biology Society, 2013, pp. 2672–2675.
-
29)
-
14. Tan, K.S., Saatchi, R., Elphick, H., et al: ‘Real-time vision based respiration monitoring system’. 2010 Int. Symp. on Communication Systems Networks and Digital Signal Processing, 2010, pp. 770–774.
-
30)
-
2. Subbe, C.P., Davies, R.G., Williams, E., et al: ‘Effect of introducing the modified early warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions’, Anaesthesia, 2003, 58, (8), pp. 797–802 (doi: 10.1046/j.1365-2044.2003.03258.x).
-
31)
-
31. Dowdall, J., Pavlidis, I.T., Tsiamyrtzis, P.: ‘Colitional tracking, computer vision and image understanding’ (Elsevier, 2007), vol. 106, pp. 205–219.
-
32)
-
12. Nakai, H., Ishihara, K., Miyake, Y., et al: ‘Non-restrictive visual respiration monitoring’. IEEE Xplore, 2000, pp. 647–651. .
-
33)
-
16. Scully, C.G., Lee, J., Meyer, J., et al: ‘Physiological parameter monitoring from optical recordings with a mobile phone’, IEEE Trans. Biomed. Eng., 2012, 59, (2), pp. 303–306 (doi: 10.1109/TBME.2011.2163157).
-
34)
-
23. Lienhart, R., Maydt, J.: ‘An extended set of Haar-like features for rapid object detection’. 2002. .
-
35)
-
6. Alkali, A.H., Saatchi, R., Elphick, H., et al: ‘Facial tracking in thermal images for real-time noncontact respiration rate monitoring’. 2013 European Modelling Symp., 2013, pp. 265–270.
-
36)
-
7. Kapur, J., Sahoo, P., Wong, A.: ‘A new method for gray level picture thresholding using the entropy of the histogram’, Comput. Graph. Image Process., 1985, 29, pp. 273–285 (doi: 10.1016/0734-189X(85)90125-2).
-
37)
-
35. Memarian, N., Chau, T., Venetsanopoulos, A.N.: ‘Application of infrared thermal imaging in rehabilitation engineering: preliminary results’. 2009 IEEE Toronto Int. Conf. in Science and Technology for Humanity, 2009, pp. 1–5.
-
38)
-
39. Young, J.W.: ‘Head and face anthropometry of adult U.S. citizens’ (Office of Aviation Medicine, Washington DC 20591, 1993). .
-
39)
-
1. McLafferty, E., Johnstone, C., Hendry, C., et al: ‘Respiratory system part 1: pulmonary ventilation’, Nurs. Stand., 2013, 27, (22), pp. 40–47 (doi: 10.7748/ns2013.01.27.22.40.e4216).
-
40)
-
28. Lucas, D., Kanade, T.: ‘An iterative image registration technique with an application to stereo vision’. Proc. Seventh Int. Joint Conf. on Artificial Intelligence, 1981, pp. 674–679. .
-
41)
-
41. Murthy, R., Pavlidis, I., Tsiamyrtzis, P.: ‘Touchless monitoring of breathing function’. Twenty-Sixth Annual Int. Conf. of the IEEE in Engineering in Medicine and Biology Society, 2004, vol. 2, pp. 1196–1199.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cds.2016.0143
Related content
content/journals/10.1049/iet-cds.2016.0143
pub_keyword,iet_inspecKeyword,pub_concept
6
6