access icon free Video anomaly detection using deep incremental slow feature analysis network

© The Institution of Engineering and Technology
Received 25/07/2015, Accepted 09/12/2015, Revised 23/11/2015, Published 01/03/2016

Existing anomaly detection (AD) approaches rely on various hand-crafted representations to represent video data and can be costly. The choice or designing of hand-crafted representation can be difficult when faced with a new dataset without prior knowledge. Motivated by feature learning, e.g. deep leaning and the ability to directly learn useful representations and model high-level abstraction from raw data, the authors investigate the possibility of using a universal approach. The objective is learning data-driven high-level representation for the task of video AD without relying on hand-crafted representation. A deep incremental slow feature analysis (D-IncSFA) network is constructed and applied to directly learning progressively abstract and global high-level representations from raw data sequence. The D-IncSFA network has the functionalities of both feature extractor and anomaly detector that make AD completion in one step. The proposed approach can precisely detect global anomaly such as crowd panic. To detect local anomaly, a set of anomaly maps, produced from the network at different scales, is used. The proposed approach is universal and convenient, working well in different types of scenarios with little human intervention and low memory and computational requirements. The advantages are validated by conducting extensive experiments on different challenge datasets.

Inspec keywords: video signal processing

Other keywords: crowd panic; video data; hand-crafted representations; deep incremental slow feature analysis network; deep leaning; D-IncSFA network; video anomaly detection; feature learning; global anomaly

Subjects: Video signal processing; Optical, image and video signal processing

References

    1. 1)
      • 35. Wang, B., Ye, M., Li, X., et al: ‘Abnormal crowd behavior detection using high-frequency and spatio-temporal features’, Mach. Vis. Appl., 2012, 23, (3), pp. 501–511.
    2. 2)
      • 6. Benezeth, Y., Jodoin, P.M., Saligrama, V., et al: ‘Abnormal events detection based on spatio-temporal co-occurences’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Miami, Florida, USA, June 2009, pp. 2458–2465.
    3. 3)
      • 21. Lee, S.C., Nevatia, R.: ‘Hierarchical abnormal event detection by real time and semi-real time multi-tasking video surveillance system’, Mach. Vis. Appl., 2014, 25, (1), pp. 133–143.
    4. 4)
      • 15. Hu, W., Xiao, X., Fu, Z., et al: ‘A system for learning statistical motion patterns’, IEEE Trans. Pattern Anal., 2006, 28, (9), pp. 1450–1464.
    5. 5)
      • 40. Yuan, Y., Fang, J., Wang, Q.: ‘Online anomaly detection in crowd scenes via structure analysis’, IEEE Trans. Cybern., 2015, 45, (3), pp. 562–575.
    6. 6)
      • 37. Wu, S., Moore, B.E., Shah, M.: ‘Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes’. Proc. Int. Conf. Computer Vision and Pattern Recognition, San Francisco, CA, USA, June 2010, pp. 2054–2060.
    7. 7)
      • 32. Sharif, M.H., Djeraba, C.: ‘An entropy approach for abnormal activities detection in video streams’, Pattern Recognit., 2012, 45, (7), pp. 2543–2561.
    8. 8)
      • 46. Shi, Y., Gao, Y., Wang, R.: ‘Real-time abnormal event detection in complicated scenes’. Proc. Int. Conf. Pattern Recognition, Istanbul, Turkey, August 2010, pp. 3653–3656.
    9. 9)
      • 38. Xiang, T., Gong, S.: ‘Video behavior pro_ling for anomaly detection’, IEEE Trans. Pattern Anal., 2008, 30, (5), pp. 893–908.
    10. 10)
      • 20. Laxhammar, R., Falkman, G.: ‘Online learning and sequential anomaly detection in trajectories’, IEEE Trans. Pattern Anal., 2014, 36, (6), pp. 1158–1173.
    11. 11)
      • 30. Reddy, V., Sanderson, C., Lovell, B.C.: ‘Improved anomaly detection in crowded scenes via cell-based analysis of foreground speed, size and texture’. Proc. Int. Conf. Computer Vision and Pattern Recognition Workshops, Colorado Springs, CO, USA, June 2011, pp. 55–61.
    12. 12)
      • 23. Li, J., Gong, S., Xiang, T.: ‘Learning behavioural context’, Int. J. Comput. Vis., 2012, 97, (3), pp. 276–304.
    13. 13)
      • 62. ‘Peds dataset’, http://www.svcl.ucsd.edu/projects/anomaly/dataset.html, accessed 2 May 2013.
    14. 14)
      • 28. Piciarelli, C., Micheloni, C., Foresti, G.L.: ‘Trajectory-based anomalous event detection’, IEEE Trans. Circuits Syst. Video Technol., 2008, 18, (11), pp. 1544–1554.
    15. 15)
      • 13. Dee, H.M., Hogg, D.: ‘Detecting inexplicable behaviour’. Proc. British Machine Vision Conf., Kingston, UK, September 2004, pp. 1–10.
    16. 16)
      • 39. Xu, D., Song, R., Wu, X., et al: ‘Video anomaly detection based on a hierarchical activity discovery within spatio-temporal contexts’, Neurocomputing, 2014, 143, pp. 144–152.
    17. 17)
      • 53. Zhang, Z., Tao, D.: ‘Slow feature analysis for human action recognition’, IEEE Trans. Pattern Anal., 2012, 34, (3), pp. 436–450.
    18. 18)
      • 65. Zafeiriou, L., Nicolaou, M., Zafeiriou, S., et al: ‘Learning slow features for behaviour analysis’. Proc. Int. Conf. Computer Vision, Sydney, Australia, December 2013, pp. 2840–2847.
    19. 19)
      • 59. ‘Unusual Crowd Activity Dataset’, http://mha.cs.umn.edu/movies/crowdactivity-all.avi, accessed July 2011.
    20. 20)
      • 54. Sun, L., Jia, K., Chan, T.H., et al: ‘Dl-sfa: Deeply learned slow feature analysis for action recognition’. Int. Conf. Computer Vision and Pattern Recognition, Columbus, OH, USA, June 2014, pp. 2625–2632.
    21. 21)
      • 11. Cong, Y., Yuan, J., Tang, Y.: ‘Video anomaly search in crowded scenes via spatio-temporal motion context’, IEEE Trans. Inf. Forensic Sec., 2013, 8, (10), pp. 1590–1599.
    22. 22)
      • 57. Hinton, G.E., Osindero, S., Teh, Y.-W.: ‘A fast learning algorithm for deep belief nets’, Neural Comput., 2006, 18, (7), pp. 1527–1554.
    23. 23)
      • 51. Berkes, P., Wiskott, L.: ‘Slow feature analysis yields a rich repertoire of complex cell properties’, J. Vis., 2005, 5, (6), p. 9.
    24. 24)
      • 55. Theriault, C., Thome, N., Cord, M.: ‘Dynamic scene classification: learning motion descriptors with slow features analysis’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Portland, OR, USA, June 2013, pp. 2603–2610.
    25. 25)
      • 10. Cong, Y., Yuan, J., Liu, J.: ‘Abnormal event detection in crowded scenes using sparse representation’, Pattern Recog., 2013, 46, (7), pp. 1851–1864.
    26. 26)
      • 9. Cong, Y., Yuan, J., Liu, J.: ‘Sparse reconstruction cost for abnormal event detection’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, June 2011, pp. 3449–3456.
    27. 27)
      • 47. Chandola, V., Banerjee, A., Kumar, V.: ‘Anomaly detection: a survey’, ACM Comput. Surv., 2009, 41, (3), p. 15.
    28. 28)
      • 29. Raghavendra, R., Bue, A.D., Cristani, M., et al: ‘Optimizing interaction force for global anomaly detection in crowded scenes’. Proc. Int. Conf. Computer Vision Workshops, Barcelona, Spain, November 2011, pp. 136–143.
    29. 29)
      • 60. ‘Anomaly detection dataset’, http://www.cse.yorku.ca/vision/research/spatiotemporal-anomalousbehavior.shtml, accessed 27 November 2013.
    30. 30)
      • 18. Kim, J., Grauman, K.: ‘Observe locally, infer globally: a space-time MRF for detecting abnormal activities with incremental updates’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Miami, Florida, USA, June 2009, pp. 2921–2928.
    31. 31)
      • 24. Li, W., Mahadevan, V., Vasconcelos, N.: ‘Anomaly detection and localization in crowded scenes’, IEEE Trans. Pattern Anal., 2014, 36, (1), pp. 18–32.
    32. 32)
      • 17. Jiang, F., Wu, Y., Katsaggelos, A.K.: ‘A dynamic hierarchical clustering method for trajectory-based unusual video event detection’, IEEE Trans. Image Process., 2009, 18, (4), pp. 907–913.
    33. 33)
      • 50. Wiskott, L., Sejnowski, T.J.: ‘Slow feature analysis: unsupervised learning of invariances’, Neural Comput., 2002, 14, (4), pp. 715–770.
    34. 34)
      • 14. Gu, X., Cui, J., Zhu, Q.: ‘Abnormal crowd behavior detection by using the particle entropy’, Optik, 2014, 125, (14), pp. 3428–3433.
    35. 35)
      • 34. Varadarajan, J., Odobez, J.M.: ‘Topic models for scene analysis and abnormality detection’. Proc. Int. Conf. Computer Vision Workshops, Kyoto, Japan, September 2009, pp. 1338–1345.
    36. 36)
      • 43. Kaltsa, V., Briassouli, A., Kompatsiaris, I., et al: ‘Swarm intelligence for detecting interesting events in crowded environments’, IEEE Trans. Iimage Process., 2015, 24, (7), pp. 2153–2166.
    37. 37)
      • 41. Zhu, X., Liu, J., Wang, J., et al: ‘Sparse representation for robust abnormality detection in crowded scenes’, Pattern Recognit., 2014, 47, (5), pp. 1791–1799.
    38. 38)
      • 3. Adam, A., Rivlin, E., Shimshoni, I., et al: ‘Robust real-time unusual event detection using multiple fixed-location monitors’, IEEE Trans. Pattern Anal., 2008, 30, (3), pp. 555–560.
    39. 39)
      • 64. Raghavendra, R., Del, Bue, A., Cristani, M., et al: ‘Optimizing interaction force for global anomaly detection in crowded scenes’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, June 2011, pp. 136–143.
    40. 40)
      • 66. Dutta, J.K., Banerjee, B.: ‘Online detection of abnormal events using incremental coding length’. Proc. 29th AAAI Conf. on Artificial Intelligence, Austin, TX, USA, January 2015.
    41. 41)
      • 8. Chan, M.T., Hoogs, A., Schmiederer, J., et al: ‘Detecting rare events in video using semantic primitives with hmm’. Proc. Int. Conf. Pattern Recognition, Cambridge, UK, August 2004, pp. 150–154.
    42. 42)
      • 63. Xu, J., Denman, S., Fookes, C., et al: ‘Unusual scene detection using distributed behaviour model and sparse representation’. Proc. Int. Conf. Advanced Video and Signal-Based Surveillance, Karlsruhe, Germany, June 2012, pp. 48–53.
    43. 43)
      • 48. Bengio, Y., Courville, A., Vincent, P.: ‘Representation learning: a review and new perspectives’, IEEE Trans. Pattern Anal., 2013, 35, (8), pp. 1798–1828.
    44. 44)
      • 25. Ma, Y., Cisar, P.: ‘Event detection using local binary pattern based dynamic textures’. Proc. Int. Conf. Computer Vision and Pattern Recognition Workshops, Miami, Florida, USA, June 2009, pp. 38–44.
    45. 45)
      • 56. LeCun, Y., Bottou, L., Bengio, Y., et al: ‘Gradient-based learning applied to document recognition’, Proc. IEEE., 1998, 86, (11), pp. 2278–2324.
    46. 46)
      • 61. ‘PETS 2009’, http://www.cvg.rdg.ac.uk/pets2009/a.html, accessed 2 June 2012.
    47. 47)
      • 36. Wang, T., Snoussi, H.: ‘Detection of abnormal visual events via global optical flow orientation histogram’, IEEE Trans. Inf. Forensic Sec., 2014, 9, (6), pp. 988–998.
    48. 48)
      • 58. Franzius, M., Wilbert, N., Wiskott, L.: ‘Invariant object recognition and pose estimation with slow feature analysis’, Neural Comput., 2011, 23, (9), pp. 2289–2323.
    49. 49)
      • 49. Kompella, V.R., Luciw, M., Schmidhuber, J.: ‘Incremental slow feature analysis: adaptive low-complexity slow feature updating from high-dimensional input streams’, Neural Comput., 2012, 24, (11), pp. 2994–3024.
    50. 50)
      • 33. Su, H., Yang, H., Zheng, S., et al: ‘The large-scale crowd behavior perception based on spatio-temporal viscous uidfield’, IEEE Trans. Inf. Forensic Sec., 2013, 8, (10), pp. 1575–1589.
    51. 51)
      • 42. Zhao, B., Fei-Fei, L., Xing, E.P.: ‘Online detection of unusual events in videos via dynamic sparse coding’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, June 2011, pp. 3313–3320.
    52. 52)
      • 26. Mahadevan, V., Li, W., Bhalodia, V., et al: ‘Anomaly detection in crowded scenes’. Proc. Int. Conf. Computer Vision and Pattern Recognition, San Francisco, CA, USA, 2010, pp. 1975–1981.
    53. 53)
      • 12. Cui, X., Liu, Q., Gao, M., et al: ‘Abnormal detection using interaction energy potentials’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Colorado Springs, CO, USA, June 2011, pp. 3161–3167.
    54. 54)
      • 31. Ren, W., Li, G., Sun, B., et al: ‘Unsupervised kernel learning for abnormal events detection’, Vis. Comput., 2015, 31, (3), pp. 245–255.
    55. 55)
      • 44. Li, N., Wu, X., Xu, D., et al: ‘Spatio-temporal context analysis within video volumes for anomalous-event detection and localization’, Neurocomputing, 2015, 155, pp. 309–319.
    56. 56)
      • 4. Anti_c, B., Ommer, B.: ‘Video parsing for abnormality detection’. Proc. Int. Conf. Computer Vision, Barcelona, Spain, November 2011, pp. 2415–2422.
    57. 57)
      • 27. Mehran, R., Oyama, A., Shah, M.: ‘Abnormal crowd behavior detection using social force model’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Miami, Florida, USA, June 2009, pp. 935–942.
    58. 58)
      • 7. Bertini, M., Del Bimbo, A., Seidenari, L.: ‘Multi-scale and real-time non-parametric approach for anomaly detection and localization’, Comput. Vis. Image Underst., 2012, 116, (3), pp. 320–329.
    59. 59)
      • 2. Krausz, B., Bauckhage, C.: ‘Loveparade 2010: automatic video analysis of a crowd disaster’, Comput. Vis. Image Underst., 2012, 116, (3), pp. 307–319.
    60. 60)
      • 1. Mishra, S., K., Bhagat, K., S.: ‘A survey on human motion detection and surveillance’, Environment, 2015, 4, (4), pp. 1044–1048.
    61. 61)
      • 16. Ihaddadene, N., Djeraba, C.: ‘Real-time crowd motion analysis’. Proc. Int. Conf. Pattern Recognition, Tempa, Folorida, USA, December 2008, pp. 1–4.
    62. 62)
      • 52. Wiskott, L., Berkes, P., Franzius, M., et al: ‘Slow feature analysis’, Scholarpedia, 2011, 6, (4), p. 5282.
    63. 63)
      • 45. Roshtkhari, M.J., Levine, M.D.: ‘An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions’, Comput. Vis. Image Underst., 2013, 117, (10), pp. 1436–1452.
    64. 64)
      • 5. Basharat, A., Gritai, A., Shah, M.: ‘Learning object motion patterns for anomaly detection and improved object detection’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Anchorage, Alaska, USA, June 2008, pp. 1–8.
    65. 65)
      • 22. Li, C., Han, Z., Ye, Q., et al: ‘Visual abnormal behavior detection based on trajectory sparse reconstruction analysis’, Neurocomputing, 2013, 119, pp. 94–100.
    66. 66)
      • 19. Kratz, L., Nishino, K.: ‘Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models’. Proc. Int. Conf. Computer Vision and Pattern Recognition, Miami, Florida, USA, June 2009, pp. 1446–1453.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2015.0271
Loading

Related content

content/journals/10.1049/iet-cvi.2015.0271
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading