Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon openaccess Assistive technology for relieving communication lumber between hearing/speech impaired and hearing people

This study proposes an automatic sign language translator, which is developed as assistive technology to help the hearing/speech impaired communities to communicate with the rest of the world. The system architecture, which includes feature extraction and recognition stages is described in detail. The signs are classified into two types: static and dynamic. Various types of sign features are presented and analysed. Recognition stage considers the hidden Markov model and segmentation signature. Real-time implementation of the system with the use of Windows7 and LINUX Fedora 16 operating systems with VMware workstation is presented in detail. The system has been successfully tested on Malaysian sign language.

References

    1. 1)
    2. 2)
    3. 3)
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
    9. 9)
    10. 10)
    11. 11)
    12. 12)
    13. 13)
    14. 14)
    15. 15)
      • 1. MFD, Malaysian Sign Language. 2012; Available at http://www.mfd.org.my/public/edu_eSign.asp.
    16. 16)
      • 28. Li, H., Greenspan, M.: ‘Continuous time-varying gesture segmentation by dynamic time warping of compound gesture models’. Int. Workshop on Human Activity Recognition and Modelling (HARAM2005), 2005, p. 8.
    17. 17)
      • 5. Grzeszczuk, R., Bradski, G., Chu, M.H., Bouguet, J.: ‘Stereo based gesture recognition invariant to 3D pose and lighting’. IEEE Conf. Computer Vision and Pattern Recognition, IEEE Computer Society, Head Island, SC, USA, 2000, vol. 1, pp. 826833.
    18. 18)
      • 36. Khan, S., Bailey, D.G., Sen Gupta, G.: ‘Delayed absolute difference (DAD) signatures of dynamic features for sign language segmentation’. Fifth Int. Conf. Automation, Robotics and Applications (ICARA2011), Wellington, New Zealand, 2011, pp. 109114.
    19. 19)
    20. 20)
      • 13. Bradski, G.R.: ‘Computer vision face tracking for use in a perceptual user interface’, Intel Technol. J., 1998, 2, (3), pp. 115.
    21. 21)
      • 33. Liang, R.-H., Ming, O.: ‘A real-time continuous gesture recognition system for sign language’. IEEE Int. Conf. Automatic Face and Gesture Recognition, Japan, 1998, pp. 558567.
    22. 22)
    23. 23)
    24. 24)
      • 7. Bilal, S., Akmeliawati, R., Momoh, J., Shafie, A.A.: ‘Dynamic approach for real-time skin detection’, J. Real-Time Image Process., 2012, p. 6.
    25. 25)
      • 24. Ong, S.C.W., Ranganath, S.: ‘A new probabilistic model for recognizing signs with systematic modulations’, Third International Workshop on Analysis and Modelling of Faces and Gestures, Rio de Janeiro, Brazil, 2007 (LNCS, 4778/2007), pp. 1630.
    26. 26)
    27. 27)
      • 37. Qt Project. Available at http://www.qt-project.org/.
    28. 28)
      • 34. Li, H., Greenspan, M.: ‘Multi-scale gesture recognition from time-varying contours’. 10th IEEE Int. Conf. Computer Vision, ICCV 2005, 2005, vol. 1, pp. 236243.
    29. 29)
      • 25. Ruiduo, Y., Sarkar, S.: ‘Detecting coarticulation in sign language using conditional random fields’. 18th Int. Conf. Pattern Recognition, 2006, ICPR 2006, 2006, vol. 2, pp. 108112.
    30. 30)
      • 4. Yang, M.H., Ahuja, N.: ‘Recognizing hand gesture using motion trajectories’. IEEE Conf. Computer Vision and Pattern Recognition, IEEE Computer Society, Fort Collins, CO, USA, 1999, vol. 1, pp. 466483.
    31. 31)
      • 12. Jusko, D.: Full Real Color Wheel Course, 2011. Available at http://www.realcolorwheel.com/human.htm.
    32. 32)
    33. 33)
      • 2. MacKenzie, I.S.: ‘Input devices and interaction techniques for advanced computing, in virtual environments and advanced interface design’ (Oxford University Press, Oxford, UK, 1995), pp. 437470.
    34. 34)
      • 29. Starner, T., Pentland, A.: ‘Real time American sign language recognition from video using hidden Markov model’. Int. Symp. Computer Vision, Florida, USA, 1995, pp. 265270.
    35. 35)
      • 23. Kahol, K., Tripathi, P., Panchanathan, S., Rikakis, T.: ‘Gesture segmentation in complex motion sequences’. IEEE Int. Conf. Automatic Face and Gesture Recognition, Seoul, Korea, 2004, vol. 3, pp. II-1058.
    36. 36)
      • 16. Segouat, J., Braffort, A.: ‘Toward modeling sign language coarticulation’, in Kopp, S., Wachsmuth, I. (Eds.): ‘Gesture in embodied communication and human–computer interaction’ (Springer Berlin Heidelberg, 2010), pp. 325336.
    37. 37)
    38. 38)
    39. 39)
    40. 40)
      • 9. Imagawa, K., Lu, S., Igi, S.: ‘Color-based hands tracking system for sign language recognition’. Third IEEE Int. Conf. Automatic Face and Gesture Recognition, Nara, Japan, 1998, pp. 462467.
    41. 41)
      • 11. Kilian, J.: Simple Image Analysis by Moments, 2001, 8 pp. Available at http://www.scribd.com/doc/39759766/Simple-Image-Analysis-by-Moments.
    42. 42)
    43. 43)
    44. 44)
    45. 45)
      • 15. Starner, T.E., Pentland, A.: ‘Real-time American sign language recognition from video using hidden Markov models’. IEEE Int. Symp. Computer Vision, Coral Gables, FL, USA, 1995, pp. 265270.
    46. 46)
      • 27. Li, H., Greenspan, M.: ‘Segmentation and recognition of continuous gestures’. IEEE Int. Conf. Image Processing, 2007, ICIP 2007, 2007, vol. 1, pp. 365368.
    47. 47)
      • 30. Vogler, C.P.: ‘American sign language recognition: reducing the complexity of the task with phoneme-based modeling and parallel hidden Markov models’. PhD dissertation, University of Pennsylvania, USA, p. 172.
    48. 48)
      • 14. Bilal, S., Akmeliawati, R., Shafie, A.A., Salami, M.J.E.: ‘Modelling of human upper body for sign language recognition’. Fifth Int. Conf. Automation, Robotics and Applications (ICARA), Wellington, New Zealand, 2011, pp. 104108.
    49. 49)
    50. 50)
      • 10. Bilal, S., Akmeliawati, R., Salami, M.J.E., Shafie, A.A., Bouhabba, E.M., et al: ‘A hybrid method using Haar-like and skin-color algorithm for hand posture detection, recognition and tracking’. Int. Conf. Mechatronics and Automation (ICMA), Xi'an, China, 2010, pp. 934939.
http://iet.metastore.ingenta.com/content/journals/10.1049/joe.2014.0039
Loading

Related content

content/journals/10.1049/joe.2014.0039
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address