Framework for dynamic hand gesture recognition using Grassmann manifold for intelligent vehicles

Framework for dynamic hand gesture recognition using Grassmann manifold for intelligent vehicles

For access to this article, please select a purchase option:

Buy article PDF
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Your details
Why are you recommending this title?
Select reason:
IET Intelligent Transport Systems — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

In this study, the authors propose a novel and robust approach to control auxiliary tasks in vehicles using hand gestures. First, they create a three-dimensional video volume by appending one frame to other that captures the motion history of frames. Then, they extract features using histogram of oriented gradients on each video volume. These features are represented in the form of subspaces on Grassmann manifold. To improve the recognition accuracy, they map the data from one manifold to another manifold with the help of a Grassmann kernel. Grassmann graph embedding discriminant analysis framework is used to classify the gestures. They perform experiments on two datasets: LISA and Cambridge Hand Gesture in three different testing methods such as 1/3-subject, 2/3-subject and cross-subject. Experimental results show that their proposed model outperforms and is comparable with the state-of-the-art methods.


    1. 1)
      • 1. Verma, B., Choudhary, A.: ‘Unsupervised learning based static hand gesture recognition from RGB-D sensor’. Int. Conf. Soft Computing and Pattern Recognition, Vellore, India, 2016, pp. 304314.
    2. 2)
      • 2. Gupta, S., Molchanov, P., Yang, X., et al: ‘Towards selecting robust hand gestures for automotive interfaces’. IEEE Intelligent Vehicles Symp. (IV), Gothenburg, Sweden, 2016, pp. 13501357.
    3. 3)
      • 3. Das, N., Ohn-Bar, E., Trivedi, M.M.: ‘On performance evaluation of driver hand detection algorithms: challenges, dataset, and metrics’. 2015 IEEE 18th Int. Conf. Intelligent Transportation Systems (ITSC), Las Palmas, Spain, 2015, pp. 29532958.
    4. 4)
      • 4. Hamm, J., Lee, D.D.: ‘Grassmann discriminant analysis: a unifying view on subspace-based learning’. Proc. 25th Int. Conf. Machine Learning, San Diego, CA, USA, 2008, vol. 4, pp. 376383.
    5. 5)
      • 5. Dalal, N., Triggs, B.: ‘Histograms of oriented gradients for human detection’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, San Diego, CA, USA, 2005, vol. 1, pp. 886893.
    6. 6)
      • 6. Harandi, M.T., Sanderson, C., Shirazi, S., et al: ‘Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), CO, USA, 2011, pp. 27052712.
    7. 7)
      • 7. Keller, J.M., Gray, M.R., Givens, J.A.: ‘A fuzzy k-nearest neighbour algorithm’, IEEE Trans. Syst. Man Cybern., 1985, pp. 580585.
    8. 8)
      • 8. Ohn-Bar, E., Trivedi, M.M.: ‘Hand gesture recognition in real time for automotive interfaces: a multimodal vision-based approach and evaluations’, IEEE Trans. Intell. Transp. Syst., 2014, 15, (6), pp. 23682377.
    9. 9)
      • 9. Kim, T.-K., Cipolla, R.: ‘Canonical correlation analysis of video volume tensors for action categorization and detection’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2009, 31, (8), pp. 14151428.
    10. 10)
      • 10. Ohn-Bar, E., Trivedi, M.: ‘In-vehicle hand activity recognition using integration of regions’. 2013 IEEE Intelligent Vehicles Symp. (IV), Gold Coast, Australia, 2013, pp. 10341039.
    11. 11)
      • 11. Ohn-Bar, E., Trivedi, M.M.: ‘A comparative study of color and depth features for hand gesture recognition in naturalistic driving settings’. IEEE 2015 Intelligent Vehicles Symp. (IV), Seoul, South Korea, 2015, pp. 845850.
    12. 12)
      • 12. Thomas, K., Magand, S., Gepperth, A., et al: ‘A light-weight real-time applicable hand gesture recognition system for automotive applications’. 2015 IEEE Intelligent Vehicles Symp. (IV), Seoul, South Korea, 2015, pp. 336342.
    13. 13)
      • 13. Dong, G., Yan, Y., Xie, M.: ‘Vision-based hand gesture recognition for human–vehicle interaction’. Proc. Int. Conf. Control, Automation and Computer Vision, 1998, pp. 151155.
    14. 14)
      • 14. Rangesh, A., Ohn-Bar, E., Trivedi, M.: ‘Long-term multi-cue tracking of hands in vehicles’, IEEE Trans. Intell. Transp. Syst., 2016, 17, (5), pp. 14831492.
    15. 15)
      • 15. Neverova, N., Wolf, C., Taylor, G.W., et al: ‘Multi-scale deep learning for gesture detection and localization’. Workshop at the European Conf. Computer Vision, Zurich, Switzerland, 2014, pp. 474490.
    16. 16)
      • 16. Molchanov, P., Gupta, S., Kim, K., et al: ‘Hand gesture recognition with 3D convolutional neural networks’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, Boston, MA, USA, 2015, pp. 17.
    17. 17)
      • 17. Deo, N., Rangesh, A., Trivedi, M.: ‘In-vehicle hand gesture recognition using hidden Markov models’. 2016 IEEE 19th Int. Conf. Intelligent Transportation Systems (ITSC), Rio de Janeiro, Brazil, 2016, pp. 21792184.
    18. 18)
      • 18. Bouchrika, T., Zaied, M., Jemai, O., et al: ‘Neural solutions to interact with computers by hand gesture recognition’, Multimedia Tools Appl., 2014, 72, (3), pp. 949975.
    19. 19)
      • 19. Bouchrika, T., Jemai, O., Zaied, M., et al: ‘Rapid and efficient hand gestures recognizer based on classes discriminator wavelet networks’. Multimedia Tools Appl., 2017, pp. 122.
    20. 20)
      • 20. Tompson, J., Stein, M., Lecun, Y., et al: ‘Real-time continuous pose recovery of human hands using convolutional networks’, ACM Trans. Graph. (ToG), 2014, 33, (5), p. 169.
    21. 21)
      • 21. Althoff, F., Lindl, R., Walchshausl, L.D.: ‘Robust multimodal hand-and-head gesture recognition for controlling automotive infotainment systems’, VDI Ber., 2005, 1919, p. 187.
    22. 22)
      • 22. Harandi, M.T., Sanderson, C., Wiliem, A., et al: ‘Kernel analysis over Riemannian manifolds for visual recognition of actions, pedestrians and textures’. 2012 IEEE Workshop on Applications of Computer Vision (WACV), Breckenridge, CO, USA, 2012, pp. 433439.
    23. 23)
      • 23. Liu, L., Shao, L.: ‘Synthesis of spatio-temporal descriptors for dynamic hand gesture recognition using genetic programming’. IEEE 2013 Tenth IEEE Int. Conf. Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China, 2013, pp. 17.
    24. 24)
      • 24. Yuan, Y., Zheng, H., Li, Z., et al: ‘Video action recognition with spatio-temporal graph embedding and spline modeling’. 2010 IEEE Int. Conf. Acoustics Speech and Signal Processing (ICASSP), Dallas, TX, USA, 2010, pp. 24222425.

Related content

This is a required field
Please enter a valid email address