access icon free Spelled sign word recognition using key frame

In this study, the authors present a new system for sign language hand gesture recognition. Using video input, the system can recognise any spelled word or alphabetic sequence signed in American Sign Language. The three main steps in the recognition process include detection of the region of interest (the hands), detection of key frames and recognition of gestures from these key frames. The proposed segmentation algorithm distinguishes regions of interest from both uniform and non-uniform backgrounds with an efficiency of 95%. The proposed key frame detection algorithm achieves an efficiency of 96.50%. A rotation-invariant algorithm for feature extraction is additionally proposed, which provides an overall gesture recognition efficiency of 84.2%.

Inspec keywords: gesture recognition; feature extraction; natural language processing

Other keywords: key frame detection algorithm; alphabetic sequence; sign language hand gesture recognition; rotation invariant algorithm; segmentation algorithm; feature extraction; American Sign Language; video input; spelled sign word recognition

Subjects: Natural language interfaces; Computer vision and image processing techniques; Image recognition

References

    1. 1)
      • 12. Bhuyan, M.K., Ghosh, D., Bora, P.K.: ‘Designing of human computer interactive platform for robotic applications’. TENCON IEEE Region, November 2005, pp. 15.
    2. 2)
      • 7. Maraqa, M., Abu-Zaiter, R.: ‘Recognition of Arabic Sign Language (ArSL) using recurrent neural networks’. IEEE First Int. Conf. on the Applications of Digital Information and Web Technologies, ICADIWT2008, pp. 47848.
    3. 3)
      • 1. Peer, P., Kovac, J., Solina, F.: ‘Human skin colour clustering for face detection’. EUROCON –Int. Conf. on Computer as a Tool’, August 2010, pp. 144148.
    4. 4)
      • 17. Murthy, G.R.S., Jadon, R.S.: ‘A review of vision based hand gestures recognition’, Int. J. Inf. Technol. Know. Manage., 2009, 2, (2), pp. 405410.
    5. 5)
      • 2. Stergiopoulou, E., Papamarkos, N.: ‘A new technique for hand gesture recognition’. Proc. of IEEE Int. Conf. on Image Processing, Atlanta, October 2006, pp. 26572660.
    6. 6)
      • 14. Kim, H.-J., Lee, J.S., Park, J.-H.: ‘Dynamic hand gesture recognition using a CNN model with 3d receptive fields’. IEEE Int. Conf. on Neural Networks and Signal Processing, Zhenjiang, China, 8–10 June 2008, pp. 1419.
    7. 7)
      • 16. Verma, R., Dev, A.: ‘Vision based hand gesture recognition using finite state machines and fuzzy logic’. IEEE Int. Conf. on Ultra Modern Telecommunications and Workshops, ICUMT, Petersburg, 2009, pp. 16.
    8. 8)
      • 9. Ghobadi, S.E., Loepprich, O.E., Ahmadov, F., Bernshausen, J., Hartmann, K., Loffeld, O.: ‘Real time hand based robot control using multimodal images’, Int. J. Comput. Sci., 2008, 35, (4), pp. 110121.
    9. 9)
      • 22. Norbert, B., Szolgay, P.: ‘Vision based human-machine interface via hand gestures’, ECCTD 2007, Seville, Spain, August2007, pp. 496499.
    10. 10)
      • 3. Rokade, R., Doye, D., Kokare, M.: ‘Hand gesture recognition by thinning method’. Proc. of IEEE Int. Conf. on Digital Image Processing, March 2009, pp. 284287.
    11. 11)
      • 15. Kim, J., Song, M.: ‘Three dimensional gesture recognition using PCA of stereo images and modified matching algorithm’. IEEE Fifth Int. Conf. on Fuzzy Systems and Knowledge Discovery, FSKD, Jinan Shandong, 2008, pp. 116120.
    12. 12)
      • 4. Xu, D.: ‘A neural network approach for hand gesture recognition in virtual reality driving training system of SPG’. 18th Int. Conf. on Pattern Recognition (ICPR'06), 2006, pp. 519522.
    13. 13)
      • 21. Bhuyan, M.K.: ‘FSM-based recognition of dynamic hand gestures via gesture summarization using key video object planes’, World Acad. Sci. Eng. Technol., 2012, 68, pp. 724735.
    14. 14)
      • 19. Gonzalez, R.C., Woods, R.E., Eddins, S.L.: ‘Digital image processing using MATLAB’ (Pearson Education, 2004), pp. 337345.
    15. 15)
      • 10. Lamberti, L., Camastra, F.: ‘Real-time hand gesture recognition using a colour glove’ (Springer-Verlag, Berlin, Heidelberg, ICIAP, 2011), pp. 365373.
    16. 16)
      • 11. Bhuyan, M.K., Ghosh, D., Bora, P.K.: ‘Finite state representation of hand gesture using key video object plane’. Proc. of IEEE Region 10 Asia-Pacific Conf., 2004, pp. 579582.
    17. 17)
    18. 18)
      • 18. Mo, S., Cheng, S., Xing, X.: ‘Hand gesture segmentation based on improved Kalman filter and TSL skin colour model’. Int. Conf. on Multimedia Technology, Hangzhou, 2011, pp. 111116.
    19. 19)
      • 20. Guo, H.: ‘Image restoration with morphological erosion and exemplar-based texture synthesis’. Sixth Int. Conf. on Wireless Communications Networking and Mobile Computing (WiCOM), 2010, pp. 14.
    20. 20)
    21. 21)
    22. 22)
      • 6. Hasan, M.M., Mishra, P.K.: ‘HSV brightness factor matching for gesture recognition system’, Int. J. Image Process., 2010, 4, (5), pp. 456467.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2012.0691
Loading

Related content

content/journals/10.1049/iet-ipr.2012.0691
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading