Boosted subunits: a framework for recognising sign language from videos

Boosted subunits: a framework for recognising sign language from videos

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Image Processing — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

This study addresses the problem of vision-based sign language recognition, which is to translate signs to English. The authors propose a fully automatic system that starts with breaking up signs into manageable subunits. A variety of spatiotemporal descriptors are extracted to form a feature vector for each subunit. Based on the obtained features, subunits are clustered to yield codebooks. A boosting algorithm is then applied to learn a subset of weak classifiers representing discriminative combinations of features and subunits, and to combine them into a strong classifier for each sign. A joint learning strategy is also adopted to share subunits across sign classes, which leads to a more efficient classification. Experimental results on real-world hand gesture videos demonstrate the proposed approach is promising to build an effective and scalable system.


    1. 1)
    2. 2)
      • Liang, R.H., Ouhyoung, M.: `A real-time continuous gesture recognition system for sign language', Proc. Third Int. Conf. Autom. Face Gesture Recognition, 1998, Nara, Japan, p. 558–565.
    3. 3)
    4. 4)
      • Coogan, T., Awad, G., Han, J., Sutherland, A.: `Real time hand gesture recognition including hand segmentation and tracking', Proc. of Int. Symposium on Visual Computing, November 2006, Nevada, USA, p. 495–504.
    5. 5)
      • Kadir, T., Bowden, R., Ong, E.J.: `Minimal training, large lexicon, unconstrained sign language recognition', Proc. of British Machine Vision Conf., September 2004, UK, p. 939–948.
    6. 6)
      • Farhadi, A., Forsyth, D., White, R.: `Transfer learning in sign language', Proc. IEEE Conf. on Computer Vision and Pattern Recognition, June 2007, Minneapolis, USA, p. 1–8.
    7. 7)
    8. 8)
      • Cooper, H., Bowden, R.: `Sign language recognition using boosted volumetric features', Proc. IAPR Conf. on Machine Vision Applications, 2007, Tokyo, p. 359–362.
    9. 9)
    10. 10)
      • S. Liddelland , R. Johnson . American sign language: the phonological base. Sign Lang. Stud. , 195 - 277
    11. 11)
    12. 12)
    13. 13)
      • Bauer, B., Kraiss, K.: `Towards an automatic sign language recognition system using subunits', Proc. of Int. Gesture Workshop on Gesture and Sign Languages in Human-Computer Interaction, April 2001, London, UK, p. 64–75.
    14. 14)
      • Fang, G., Gao, X., Gao, W., Chen, Y.: `A novel approach to automatically extracting basic units from Chinese Sign Language', Proc. Int. Conf. Pattern Recognition, August 2004, UK, p. 454–457.
    15. 15)
      • Zhang, L.: `Recognition of sign language subwords based on boosted hidden Markov models', Proc. Int. Conf. on Multimodal Interfaces, 2005, p. 282–287.
    16. 16)
      • Theodorakis, S., Pitsikalis, V., Maragos, P.: `Model-level data-driven sub-units for signs in videos of continuous sign language', Proc. ICASSP Texas, 2010, USA, p. 2262–2265.
    17. 17)
      • Cooper, H., Bowden, R.: `Sign language recognition using linguistically derived sub-units', Proc. Fourth Workshop on the Representation and Processing of Sign Languages: Corpora and Sign Language Technologies, 2010, Valetta, Malta, p. 57–61.
    18. 18)
      • Ong, E., Bowden, R.: `Detection and segmentation of hand shapes using boosted classifiers', Proc. Sixth Int. Conf. on Automatic Face and Gesture Recognition, 2004, Korea, p. 889–894.
    19. 19)
      • Awad, G., Han, J., Sutherland, A.: `Novel boosting framework for subunit-based sign language recognition', Proc. of IEEE Int. Conf. on Image Processing, November 2009, Cairo, Egypt, p. 2729–2732.
    20. 20)
    21. 21)
    22. 22)
      • M.-K. Hu . Visual pattern recognition by moment invariants. IRE Trans. Inf. Theory , 179 - 187
    23. 23)
      • Sakoe, H., Chiba, S.: `A dynamic programming approach to continuous speech recognition', Proc. Int. Congress on Acoustics, August 1971, Budapest, Hungary, p. 65–68.
    24. 24)
      • Viola, P., Jones, M.: `Rapid object detection using a boosted cascade of simple classifiers', Proc. of Int. Conf. on Computer Vision and Pattern Recognition, December 2001, Hawaii, USA, p. 511–518.
    25. 25)
    26. 26)
      • Cooper, H., Bowden, R.: `Sign language recognition: working with Limited Corpora', Proc. of the Fifth Int. Conf. on Universal Access in Human-Computer Interaction, 2009, p. 472–481.
    27. 27)
    28. 28)
      • Bautista, M., Hernández-Vela, A., Escalera, S.: `Probability-based dynamic time warping for gesture recognition on RGB-D data', Int. Workshop on Depth Image Analysis, 2012.
    29. 29)
      • Escalera, S., Escudero, A., Radeva, P., Vitrià, J.: `Adaptive dynamic space time warping for real time sign language recognition', Fourth CVC Workshop on Research and Development: New Trends and Challenges in Computer Vision, 2009, p. 155–160.
    30. 30)
      • Gao, W., Fang, G., Zhao, D., Chen, Y.: `Transition movement models for large vocabulary continuous sign language recognition', Int. Conf. on Automatic Face and Gesture Recognition, 2004, p. 553–558.

Related content

This is a required field
Please enter a valid email address