access icon free Human action recognition using similarity degree between postures and spectral learning

In recent years, there has been renewed interest in developing methods for skeleton-based human action recognition. In this study, the challenging problem of the similarity degree of skeleton-based human postures is addressed. Human posture is described by screw motions between 3D rigid bodies, which can be seen as a relation matrix of 3D rigid bodies (RMRB3D). A linear subspace, a point of a Grassmannian manifold, is spanned by the orthonormal basis of matrix RMRB3D. A powerful way to compute the similarity degree between postures is researched to solve the geodesic distance between points on the Grassmannian manifold. Then representative postures are extracted through spectral clustering over representative postures. An action will be represented by a symbol sequence generated with a global linear eigenfunction constructed by spectral embedding. Finally, dynamic time warping and hidden Markov model (HMM) are used to classify these action sequences. The experimental evaluations of the proposed method on several challenging 3D action datasets show that the proposed approaches achieve promising results compared with other skeleton-based human action recognition algorithms.

Inspec keywords: pattern clustering; hidden Markov models; image recognition; image sequences; eigenvalues and eigenfunctions; learning (artificial intelligence); feature extraction; image motion analysis; matrix algebra

Other keywords: hidden Markov model; geodesic distance; Grassmannian manifold; skeleton-based human posture similarity degree; spectral embedding; human action recognition; screw motions; HMM; relation matrix of 3D rigid bodies; global linear eigenfunction; spectral learning; symbol sequence; matrix RMRB3D orthonormal basis; skeleton-based human action recognition; representative postures; spectral clustering; action sequences; dynamic time warping

Subjects: Knowledge engineering techniques; Markov processes; Linear algebra (numerical analysis); Computer vision and image processing techniques; Image recognition; Linear algebra (numerical analysis); Markov processes

References

    1. 1)
      • 9. Ding, W., Liu, K., Li, G., et al: ‘Human action recognition using spectral embedding to similarity degree between postures’. Visual Communications and Image Processing (VCIP), Chengdu, China, November 2016, pp. 14.
    2. 2)
      • 11. Presti, L.L., La Cascia, M.: ‘3D skeleton-based human action classification: a survey’, Pattern Recognit., 2016, 53, (3), pp. 130147.
    3. 3)
      • 13. Yang, X., Zhang, C., Tian, Y.: ‘Recognizing actions using depth motion maps-based histograms of oriented gradients’. Proc. 20th ACM Int. Conf. Multimedia, ACM, Nara, Japan, October 29–November 2, 2012, pp. 10571060.
    4. 4)
      • 24. Schwarz, L.A., Mateus, D.: ‘Manifold learning for ToF-based human body tracking and activity recognition’, Proc. British Machine Vision Conf., Aberystwyth, UK, September 2010, pp. 111.
    5. 5)
      • 36. Luo, J., Wang, W., Qi, H.: ‘Group sparsity and geometry constrained dictionary learning for action recognition from depth maps’. Proc. IEEE Int. Conf. Computer Vision, Sydney, NSW, Australia, December 2013, pp. 18091816.
    6. 6)
      • 23. Gong, D., Medioni, G.: ‘Dynamic manifold warping for view invariant action recognition’. 2011 IEEE Int. Conf. Computer Vision (ICCV), Barcelona, Spain, November 2011, pp. 571578.
    7. 7)
      • 39. Wang, L., Zhang, J., Zhou, L.: ‘Beyond covariance: feature representation with nonlinear kernel matrices’. Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, December 2015, pp. 45704578.
    8. 8)
      • 15. Chen, C., Jafari, R., Kehtarnavaz, N.: ‘Improving human action recognition using fusion of depth camera and inertial sensors’, IEEE Trans. Human-Machine Syst., 2015, 45, (1), pp. 5161.
    9. 9)
      • 31. Harandi, M.T., Sanderson, C., Shirazi, S., et al: ‘Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, June 2011, pp. 27052712.
    10. 10)
      • 16. Chen, C., Liu, M., Zhang, B.: ‘3D action recognition using multi-temporal depth motion maps and fisher vector’. Proc. Int. Joint Conf. Artificial Intelligence.2016, pp. 33313337.
    11. 11)
      • 37. Wang, J., Liu, Z., Wu, Y.: ‘Mining actionlet ensemble for action recognition with depth cameras’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, June 2012, pp. 12901297.
    12. 12)
      • 7. Yao, A., Gall, J., Fanelli, G.: ‘Does human action recognition benefit from pose estimation?’. Proc. 22nd British Machine Vision Conf. (BMVC), Dundee, Scotland, 29 August–2 September 2011, pp. 67.167.11.
    13. 13)
      • 27. Hall, B.: ‘Lie groups, Lie algebras, and representations: an elementary introduction’ (Springer, 2004, 2015, 2nd edn).
    14. 14)
      • 25. Bloom, V., Makris, D., Argyriou, V.: ‘Clustered spatio-temporal manifolds for online action recognition’. 22nd Int. Conf. Pattern Recognition (ICPR), Stockholm, Sweden, August 2014. pp. 39633968.
    15. 15)
      • 18. Li, W., Zhang, Z., Liu, Z.: ‘Action recognition based on a bag of 3d points’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA, USA, June, 2010, pp. 914.
    16. 16)
      • 6. Han, J., Shao, L., Xu, D., et al: ‘Enhanced computer vision with Microsoft Kinect sensor: a review’, IEEE Trans. Cybern., 2013, 43, (5), pp. 13181334.
    17. 17)
      • 10. Liang, B., Zheng, L.: ‘A survey on human action recognition using depth sensors’. Int. Conf. Digital Image Computing: Techniques and Applications (DICTA), Adelaide, SA, Australia, November 2015, pp. 18.
    18. 18)
      • 28. Pennec, X., Fillard, P., Ayache, N.: ‘A Riemannian framework for tensor computing’, Int. J. Computer Vision, 2006, 66, (1), pp. 4166.
    19. 19)
      • 32. Von Luxburg, U.: ‘A tutorial on spectral clustering’, Stat. Comput., 2007, 17, (4), pp. 395416.
    20. 20)
      • 35. Vemulapalli, R., Arrate, F., Chellappa, R.: ‘Human action recognition by representing 3d skeletons as points in a lie group’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, June 2014, pp. 588595.
    21. 21)
      • 38. Seidenari, L., Varano, V., Berretti, S.: ‘Recognizing actions from depth cameras as weakly aligned multi-part bag-of-poses’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, Portland, OR, USA, June 2013, pp. 479485.
    22. 22)
      • 30. Golub, G.H., Van Loan, C.F.: ‘Matrix computations’ (Johns Hopkins University Press, Baltimore, MD, USA, 1996), pp. 374426.
    23. 23)
      • 19. Xia, L., Chen, C., Aggarwal, J.: ‘View invariant human action recognition using histograms of 3d joints’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition Workshops (CVPRW), Providence, RI, USA, June, 2012, pp. 2027.
    24. 24)
      • 20. Yang, X.: ‘Effective 3d action recognition using eigenjoints’, J. Vis. Commun. Image Represent., 2014, 25, (1), pp. 211.
    25. 25)
      • 14. Xia, L., Aggarwal, J.K.: ‘Spatio-temporal depth cuboid similarity feature for activity recognition using depth camera’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Portland, Oregon, June, 2013, pp. 28342841.
    26. 26)
      • 12. Wang, J., Liu, Z., Chorowski, J.: ‘Robust 3D action recognition with random occupancy patterns’. Computer Vision-ECCV, Florence, Italy, October, 7–13, 2012, pp. 872885.
    27. 27)
      • 3. Jalal, A., Uddin, M.Z.: ‘Depth video-based human activity recognition system using translation and scaling invariant features for life logging at smart home’, IEEE Trans. Consum. Electron., 2012, 58, (3), pp. 863871.
    28. 28)
      • 29. Edelman, A., Arias, T.A., Smith, S.T.: ‘The geometry of algorithms with orthogonality constraints’, SIAM J. Matrix Anal. Appl., 1998, 20, (2), pp. 303353.
    29. 29)
      • 5. Shotton, J., Sharp, T., Kipman, A., et al: ‘Real-time human pose recognition in parts from single depth images’, Commun. ACM, 2013, 56, (1), pp. 116124.
    30. 30)
      • 17. Zhang, B., Yang, Y., Chen, C.: ‘Action recognition using 3D histograms of texture and a multi-class boosting classifier’, IEEE Trans. Image Process., 2017, 26, (10), pp. 46484660.
    31. 31)
      • 34. Ding, W., Liu, K., Fu, X., et al: ‘Profile HMMs for skeleton-based human action recognition’, Signal Process. Image Commun., 2016, 42, (3), pp. 109119.
    32. 32)
      • 2. Lao, W., Han, J.: ‘Automatic video-based human motion analyzer for consumer surveillance system’, IEEE Trans. Consum. Electron., 2009, 55, (2), pp. 591598.
    33. 33)
      • 33. Ng, A.Y., Jordan, M.I., Weiss, Y.: ‘On spectral clustering: analysis and an algorithm’, in Dietterich, T.G., Becker, S., Ghahramani, Z. (Eds.): ‘Advances in Neural Information Processing Systems (NIPS)’ (Cambridge: MIT Press, 1st edn, 2002), pp. 894856.
    34. 34)
      • 22. Slama, R., Wannous, H., Daoudi, M.: ‘Accurate 3D action recognition using learning on the Grassmann manifold’, Pattern Recognit., 2015, 48, (2), pp. 556567.
    35. 35)
      • 8. Park, F.C.: ‘Computational aspects of the product-of-exponentials formula for robot kinematics’, IEEE Trans. Autom. Control, 1994, 39, (3), pp. 643647.
    36. 36)
      • 4. Aggarwal, J.K., Xia, L.: ‘Human activity recognition from 3d data: a review’, Pattern Recognit. Lett., 2014, 48, (10), pp. 7080.
    37. 37)
      • 1. Fothergill, S., Mentis, H., Kohli, P., et al: ‘Instructing people for training gestural interactive systems’. Proc. SIGCHI Conf. Human Factors in Computing Systems, ACM, Austin, Texas, USA, May 5–10, 2012, pp. 17371746.
    38. 38)
      • 21. Ellis, C., Masood, S.Z., Tappen, M.F.: ‘Exploring the trade-off between accuracy and observational latency in action recognition’, Int. J. Comput. Vision, 2013, 101, (3), pp. 420436.
    39. 39)
      • 26. Lui, Y.M., Beveridge, J.R.: ‘Image-set matching using a geodesic distance and cohort normalization’. 8th IEEE Int. Conf. Automatic Face Gesture Recognition, Amsterdam, Netherlands, September 2008, pp. 16.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2017.0031
Loading

Related content

content/journals/10.1049/iet-cvi.2017.0031
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading