access icon free Body posture graph: a new graph-based posture descriptor for human behaviour recognition

Behaviour recognition and analysis is one of the most challenging problems in video processing. In this study, the authors propose a new graph-based representation of body silhouette by means of non-linear mixture modelling. Firstly, the authors suppose the body silhouette as an objective function and try to approximate it by a set of elliptical basis functions (EBFs). By using parameters of these learned EBF kernels, vertices and edges of a graph are created. Since this graph is highly matched with the real skeleton of the body silhouette and represents the posture, they name it body posture graph (BPG). Then a posture descriptor is constructed by sorting the BPG's vertices according to a position-dependent ordering algorithm. Thus, the descriptor contains not only body limbs connectivity information but also the spatial information. Therefore the descriptor is very effective for body posture description and is accurate for behaviour recognition purposes. They use simple fully-connected hidden Mrakov model to learn and classify the sequences of postures. Good performance of the proposed features was proved by results of various and numerous experiments which were implemented on three different datasets: Sinica Academia, KTH (Kungliga Tekniska Högskolan) and UCF (University of Central Florida) Sports Action.

Inspec keywords: video signal processing; hidden Markov models; image representation; graph theory; image classification; pose estimation; image sequences

Other keywords: learned EBF kernels; objective function; KTH dataset; elliptical basis functions; human behaviour recognition; posture sequence classification; body posture description; Sinica Academia datasets; position-dependent ordering algorithm; BPG vertices; UCF sport action datasets; fully-connected hidden Mrakov model; body posture graph; nonlinear mixture modelling; body silhouette; EBF; graph-based representation; graph-based posture descriptor; body limb connectivity information; video processing

Subjects: Image recognition; Combinatorial mathematics; Markov processes; Markov processes; Video signal processing; Combinatorial mathematics

References

    1. 1)
      • 18. Aminian-Modarres, A.F., Soryani, M.: ‘Extraction of a new semantic feature based on image segmentation for human behaviour recognition’. Proc. Conf. Machine Vision and Image Processing (MVIP), Tehran, Iran, November 2011, (in Persian).
    2. 2)
      • 24. Zhang, S., Yao, H., Liu, S.: ‘Dynamic background subtraction based on local dependency histogram’, Int. J. Pattern Recogn. Artif. Intell., 2009, 23, (7), pp. 13971419 (doi: 10.1142/S0218001409007569).
    3. 3)
      • 31. Monti, F., Regazzoni, C.S.: ‘Human action recognition using the motion of interest points’. Proc. Int. Conf. Image Processing (ICIP), Hong Kong, China, September 2010, pp. 709712.
    4. 4)
      • 2. Marr, D., Nishihara, H.K.: ‘Representation and recognition of the spatial organization of three-dimensional shapes’, Proc. Royal Soc. London, Series B., Biological Sci., 1978, 200, (1140), pp. 269294 (doi: 10.1098/rspb.1978.0020).
    5. 5)
      • 11. Yilmaz, A., Shah, M.: ‘Actions sketch: a novel action representation’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Diego, USA, June 2005, pp. 984989.
    6. 6)
      • 17. Hsieh, J.W., Chuang, C.H., Chen, S.Y., Chen, C.C., Fan, K.C.: ‘Segmentation of human body parts using deformable triangulation’, IEEE Trans. Syst. Man Cyber. A Syst. Humans, 2010, 40, (3), pp. 596610 (doi: 10.1109/TSMCA.2010.2040272).
    7. 7)
      • 1. Weinland, D., Ronfard, R., Boyer, E.: ‘A survey of vision-based methods for action representation, segmentation and recognition’, Comput. Vis. Image Underst., 2011, 115, (2), pp. 224241 (doi: 10.1016/j.cviu.2010.10.002).
    8. 8)
      • 23. Zhou, J., Hoang, J.: ‘Real time robust human detection and tracking system’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition Workshop, June 2005, pp. 149.
    9. 9)
      • 8. Wilson, A.D., Bobick, A.F.: ‘Parametric hidden Markov models for gesture recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 1999, 21, (9), pp. 884900 (doi: 10.1109/34.790429).
    10. 10)
      • 14. Yu, E., Aggarwal, J.K.: ‘Human action recognition with extremities as semantic posture representation’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition, Miami, USA, June 2009.
    11. 11)
      • 28. Schüldt, C., Laptev, I., Caputo, B.: ‘Recognizing human actions: a local SVM approach’. Proc. Int. Conf. Pattern Recognition, Cambridge, UK, August 2004, pp. 3236.
    12. 12)
      • 25. Gonzalez, R.C., Woods, R.E.: ‘Digital image processing’ (Prentice-Hall, 2008, 3rd edn.).
    13. 13)
      • 4. Rohr, K.: ‘Towards model-based recognition of human movements in image sequences’, CVGIP: Image Underst., 1994, 59, (1), pp. 94115 (doi: 10.1006/ciun.1994.1006).
    14. 14)
      • 30. UCF Sports Action Data Set, available at: http://www.crcv.ucf.edu/data/UCF_Sports_Action.php, accessed April 2012.
    15. 15)
      • 27. Sinica Academia Dataset, available at: http://www.iis.sinica.edu.tw/~ulin/Behavior, accessed April 2012.
    16. 16)
      • 9. Yamato, J., Ohya, J., Ishii, K.: ‘Recognizing human action in time-sequential images using hidden Markov model’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Champaign, USA, June 1992, pp. 379385.
    17. 17)
      • 35. Wong, S.F., Kim, T.K., Cipolla, R.: ‘Learning motion categories using both semantic and structural information’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition, Minneapolis, USA, June 2007.
    18. 18)
      • 12. Shao, L., Ji, L., Liu, Y., Zhang, J.: ‘Human action segmentation and recognition via motion and shape analysis’, Pattern Recogn. Lett., 2012, 33, (4), pp. 438445 (doi: 10.1016/j.patrec.2011.05.015).
    19. 19)
      • 20. Kumar, M., Yadav, N.: ‘Multilayer perceptrons and radial basis function neural network methods for the solution of differential equations: a survey’, Comput. Math. Appl., 2011, 62, (10), pp. 37963811 (doi: 10.1016/j.camwa.2011.09.028).
    20. 20)
      • 7. Brand, M., Oliver, N., Pentland, A.: ‘Coupled hidden Markov models for complex action recognition’. Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition, San-Juan, June 1997, pp. 994999.
    21. 21)
      • 22. Luo, J.C., Ming, D.P., Shen, Z.F., Chen, Q.X., Zheng, J.: ‘Elliptical basis function network for classification of remote-sensing images’, Shuju Caiji Yu Chuli/J. Data Acquisition Process., 2005, 20, (1), pp. 812.
    22. 22)
      • 32. Wang, Y., Mori, G.: ‘Human action recognition by semi-latent topic models’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (10), pp. 17621774 (doi: 10.1109/TPAMI.2009.43).
    23. 23)
      • 3. Hogg, D.: ‘Model-based vision: a program to see a walking person’, Image Vis. Comput., 1983, 1, (1), pp. 520 (doi: 10.1016/0262-8856(83)90003-3).
    24. 24)
      • 34. Schindler, K., Van-Gool, L.: ‘Action snippets: how many frames does human action recognition require?’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, USA, June 2008.
    25. 25)
      • 5. Ramanan, D., Forsyth, D.A.: ‘Automatic annotation of everyday movements’. Technical report, (UC Berkeley, 2003).
    26. 26)
      • 36. Jhuang, H., Serre, T., Wolf, L., Poggio, T.: ‘A biologically inspired system for action recognition’. Proc. Int. IEEE Conf. Computer Vision, Rio de Janeiro, Brazil, October 2007.
    27. 27)
      • 10. Bobick, A.F., Davis, J.W.: ‘The recognition of human movement using temporal templates’, IEEE Trans. Pattern Anal. Mach. Intell., 2001, 23, (3), pp. 257267 (doi: 10.1109/34.910878).
    28. 28)
      • 16. Yu, E., Aggarwal, J.K.: ‘Detection of fence climbing from monocular video’. Proc. Int. Conf. Pattern Recognition, Hong Kong, China, August 2006, pp. 375378.
    29. 29)
      • 15. Fujiyoshi, H., Lipton, A.: ‘Real-time human motion analysis by image skeletonization’. IEEE Workshop. Applications of Computer Vision, Princeton, NJ, August 1998, pp. 1521.
    30. 30)
      • 21. Buhmann, M.D.: ‘Radial basis functions: theory and implementations’. PhD thesis, Cambridge University Press, 2003.
    31. 31)
      • 19. Aminian-Modarres, A.F., Soryani, M.: ‘A semantic posture representation based on extraction of silhouette indicator points for human action recognition’. Proc. Conf. Biennial Broadcast Engineering Conf. (BEC), Tehran, Iran, December 2011, (in Persian).
    32. 32)
      • 33. Liu, J., Shah, M.: ‘Learning human actions via information maximization’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, USA, June 2008.
    33. 33)
      • 6. Campbell, L.W., Becker, D.A., Azarbayejani, A., Bobick, A.F., Pentland, A.: ‘Invariant features for 3-D gesture recognition’. Proc. Int. Conf. Automatic Face and Gesture Recognition, Killington, VT, October 1996, pp. 157162.
    34. 34)
      • 13. Ding, J., Ye, M.: ‘Human action recognition: a novel application of the stochastic detector enriched by body-parts information’, J. Inf. Comput. Sci., 2011, 8, (10), pp. 18731880.
    35. 35)
      • 29. Rodriguez, M.D., Ahmed, J., Shah, M.: ‘Action MACH: a Spatio-temporal maximum average correlation height filter for action recognition’. Proc. Int. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, USA, June 2008.
    36. 36)
      • 26. Liang, Y., Shih, S., Shih, A., Liao, H., Lin, C.: ‘Learning atomic human actions using variable-length Markov models’, IEEE Trans. Syst. Man Cybern., 2009, Part B, 39, (1), pp. 268280 (doi: 10.1109/TSMCB.2008.2005643).
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2012.0121
Loading

Related content

content/journals/10.1049/iet-cvi.2012.0121
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading