The significant progress in visual surveillance has been motivated by the need to emulate some of the human ability to monitor activity in human-made environments, particularly in the contexts of security and safety. The rapid rise in numbers of cameras installed in public and private places makes such automation desirable, at least to reduce CCTV workload. Real-world applications of visual surveillance impose the need of robust real-time solutions, able to deal with a wide range of circumstances and environmental conditions. Conventional approaches work based on what has become known as motion (or change) detection followed by tracking (in single or multiple camera systems). Objects of interest are represented by rectangular blobs and decisions on whether something might be interesting are made on rules or learned patterns of presence and trajectories of such blobs. There is growing interest in looking ‘inside the box’ for applications that are concerned with detailed human activity recognition and with robust detection of people even when image backgrounds change, as is the case of a moving camera. In this study, the authors consider the general problem of robust pedestrian detection irrespective of background, reviewing the state of the art, showing some representative results and suggesting ways forward.

References

1. 1)
  - Wu, B., Nevatia, R.: `Detection of multiple, partially occluded humans in a single image by Bayesian combination of edgelet part detectors', IEEE 10th Int. Conf. on Computer Vision, 2005.
2. 2)
  - S. Avidan . Ensemble tracking. IEEE Trans. Pattern Anal. Mach. Intell. , 2 , 261 - 271
3. 3)
  - Leibe, B., Schiele, B.: `Scale-invariant object categorization using a scale-adaptive mean-shift search', DAGM, 2004, p. 145–153.
4. 4)
  - Huang, C., Ai, H., Li, Y., Lao, S.: `Vector boosting for rotation invariant multi-view face detection', Proc. 10th IEEE Int. Conf. on Computer Vis., 2005, vol, 1, p. 446–453.
5. 5)
  - Förstner, W., Moonen, B.: `A metric for covariance matrices', Technical, 1999.
6. 6)
  - Felzenszwalb, P., McAllester, D., Ramanan, D.: `A discriminatively trained, multiscale, deformable part model', Conf. on Computer Vision and Pattern Recognition Citeseer, 2008.
7. 7)
  - Dalal, N., Triggs, B., Schmid, C.: `Human detection using oriented histograms of flow and appearance', The Ninth European Conf. on Computer Vision, 2006, 3952, p. 428.
8. 8)
  - Duan, G., Ai, H., Lao, S.: `A structural filter approach to human detection', ECCV, 2010, p. 238–251.
9. 9)
  - Tu, Z.: `Probabilistic boosting-tree: learning discriminative models for classification, recognition, and clustering', IEEE 10th Int. Conf. on Computer Vision, 2005, 3.
10. 10)
  - Mahlisch, M., Oberlander, M., Lohlein, O., Gavrila, D., Ritter, W.: `A multiple detector approach to low-resolution fir pedestrian recognition', IEEE Intelligent Vehicles Symp., 2005, p. 325–330.
11. 11)
  - G. Mori , S. Belongie , J. Malik . Efficient shape matching using shape contexts. Pattern Anal. Mach. Intell. , 11 , 1832 - 1837
12. 12)
  - Ahonen, T., Hadid, A., Pietikainen, M.: `Face recognition with local binary patterns', Eighth European Conf. on Computer Vision, 2004, p. 469–481.
13. 13)
  - Mikolajczyk, K., Schmid, C.: `Indexing based on scale invariant interest points', Proc. Eigth Int. Conf. on Computer Vision, 2001, vol, 1, p. 525–531.
14. 14)
  - Huang, C., Al, H., Wu, B., Lao, S.: `Boosting nested cascade detector for multi-view face detection', Proc. 17th Int. Conf. on Pattern Recognition, 2004, 2.
15. 15)
  - D.M. Gavrila , S. Munder . Multi-cue pedestrian detection and tracking from a moving vehicle. Int. J. Comput. Vis. , 1 , 41 - 59
16. 16)
  - Harris, C., Stephens, M.: `A combined corner and edge detector', Alvey Vision Conf., 1988, 15, p. 50.
17. 17)
  - Zhang, L., Nevatia, R.: `Efficient scan-window based object detection using gpgpu', Conf. on Computer Vision and Pattern Recognition, 2008.
18. 18)
  - Ess, A., Leibe, B., Van Gool, L.: `Depth and appearance for mobile scene analysis', ICCV, 2007.
19. 19)
  - Tuzel, O., Porikli, F., Meer, P.: `Human detection via classification on riemannian manifolds', Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2007, 1.
20. 20)
  - Freund, Y., Schapire, R.E.: `Experiments with a new boosting algorithm', The 13th Conf. on Machine Learning, 1996, p. 148–156.
21. 21)
  - Schwartz, W.R., Kembhavi, A., Harwood, D., Davis, L.S.: `Human detection using partial least squares analysis', IEEE 12th Int. Conf. on Computer Vision, 2009, p. 24–31.
22. 22)
  - Grabner, H., Bischof, H.: `On-line boosting and vision', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2006, vol, 1, p. 260–267.
23. 23)
  - Xiao, R., Zhu, L., Zhang, H.J.: `Boosting chain learning for object detection', Proc. ICCV, 2003, 1, p. 709–715.
24. 24)
  - Dollár, P., Tu, Z., Perona, P., Belongie, S.: `Integral channel features', British Machine Vision Conf., 2009.
25. 25)
  - T. Ojala , M. Pietikǎinen , D. Harwood . A comparative study of texture measures with classification based on featured distributions. Pattern Recognit. , 1 , 51 - 59
26. 26)
  - A.K. Jain , R.P.W. Duin , J. Mao . Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Mach. Intell. , 1 , 4 - 37
27. 27)
  - C. Schmid , Roger Mohr . Local grayvalue invariants for image retrieval. IEEE Trans. Pattern Anal. Mach. Intell. , 5 , 530 - 535
28. 28)
  - T. Lindeberg . Feature detection with automatic scale selection. Int. J. Comput. Vis. , 2 , 79 - 116
29. 29)
  - J. Friedman , T. Hastie , R. Tibshirani . Additive logistic regression: a statistical view of boosting. Ann. Stat. , 2 , 337 - 407
30. 30)
  - D.P. Huttenlocher , G.A. Klanderman , W.J. Rucklidge . Comparing images using the Hausdorff distance. IEEE Trans. Pattern Anal. Mach. Intell. , 9 , 850 - 863
31. 31)
  - O'Sullivan, J., Langford, J., Caruana, R., Blum, A.: `Featureboost: A meta-learning algorithm that improves model robustness', Proc. 17th Int. Conf. on Machine Learning, 2000, p. 703–710.
32. 32)
  - Viola, P., Jones, M.: `Rapid object detection using a boosted cascade of simple', IEEE Conf. on Computer Vision and Pattern Recognition, 2001.
33. 33)
  - Maji, S., Berg, A.C., Malik, J.: `Classification using intersection kernel support vector machines is efficient', IEEE Conf. on Computer Vision and Pattern Recognition, CVPR 2008, 2008, p. 1–8.
34. 34)
  - P. Dollár , C. Wojek , B. Schiele , P. Perona . Pedestrian detection: an evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. , 4 , 743 - 761
35. 35)
  - Dalal, N., Triggs, B.: `Histograms of oriented gradients for human detection', IEEE Conf. on Computer Vision and Pattern Recognition, 2005, 1, p. 886.
36. 36)
  - R. Nevatia , B. Wu . Detection and tracking of multiple, partially occluded humans by Bayesian combination of edgelet based part detectors. Int. J. Comput. Vis. , 2 , 247 - 266
37. 37)
  - S. Belongie , J. Malik , J. Puzicha . Shape matching and object recognition using shape contexts. IEEE Trans. Pattern Anal. Mach. Intell. , 4 , 509 - 522
38. 38)
  - Mu, Y., Yan, S., Liu, Y., Huang, T., Zhou, B.: `Discriminative local binary patterns for human detection in personal album', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2008, p. 1–8.
39. 39)
  - Huang, C., Ai, H., Li, Y., Lao, S.: `Learning sparse features in granular space for multi-view face detection', Proc. Seventh Int. Conf. on Automatic Face and Gesture Recognition, 2006, p. 401–407.
40. 40)
  - Dollár, P., Tu, Z., Tao, H., Belongie, S.: `Feature mining for image classification', Conf. on Computer Vision and Pattern Recognition, 2007, p. 1–8.
41. 41)
  - P. Viola , M. Jones . Robust real-time face detection. Int. J. Comput. Vis. , 2 , 137 - 154
42. 42)
  - Duan, G., Huang, C., Ai, H., Lao, S.: `Boosting associated pairing comparison features for pedestrian detection', IEEE 12th Int. Conf. on Computer Vision Workshops (ICCV Workshops), 2009, p. 1097–1104.
43. 43)
  - Opelt, A., Pinz, A., Zisserman, A.: `Incremental learning of object detectors using a visual shape alphabet', IEEE Conf. on Computer Vision and Pattern Recognition, 2006, 1, p. 4.
44. 44)
  - Zhang, W., Zelinsky, G., Samaras, D.: `Real-time accurate object detection using multiple resolutions', Proc. Int. Conf. on Computer Vision, 2007.
45. 45)
  - Wojek, C., Walk, S., Schiele, B.: `Multi-cue onboard pedestrian detection', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2009.
46. 46)
  - Wu, B., Nevatia, R.: `Optimizing discrimination-efficiency tradeoff in integrating heterogeneous local features for object detection', IEEE Conf. on Computer Vision and Pattern Recognition, 23–28 June 2008, p. 1–8.
47. 47)
  - B. Leibe , A. Leonardis , B. Schiele . Robust object detection with interleaved categorization and segmentation. Int. J. Computer Vision , 1 , 259 - 289
48. 48)
  - R.E. Schapire . The boosting approach to machine learning: an overview. Lect. Notes Stat. , 149 - 172
49. 49)
  - G. Zhao , M. Pietikäinen . Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Patt. Anal. Mach. Intell. , 6 , 915 - 928
50. 50)
  - Babenko, B., Dollár, P., Tu, Z., Belongie, S.: `Simultaneous learning and alignment: multi-instance and multi-pose learning', 10thEuropean Conf. on Computer Vision, 2008.
51. 51)
  - Bar-Hillel, A., Levi, D., Krupka, E., Goldberg, C.: `Part-based feature synthesis for human detection', ECCV, 2010, p. 127–142.
52. 52)
  - Leibe, B., Cornelis, N., Cornelis, K., Van Gool, L.: `Dynamic 3d scene analysis from a moving vehicle', Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2007.
53. 53)
  - R. Rosipal , N. Kramer . Overview and recent advances in partial least squares. Lect. Notes Comput. Sci.
54. 54)
  - Simonnet, D., Velastin, S.A., Orwell, J., Turkbeyler, E.: `Selecting and evaluating data for training a pedestrian detector for crowded conditions', IEEE Int. Conf. on Signal and Image Processing Applications, 2011.
55. 55)
  - Seemann, E., Leibe, B., Mikolajczyk, K., Schiele, B.: `An evaluation of local shape-based features for pedestrian detection', British Machine Vision Conf., 2005.
56. 56)
  - Jones, M., Viola, P., Jones, M.J., Snow, D.: `Detecting pedestrians using patterns of motion and appearance', Int. Conf. on Computer Vision, 2003.
57. 57)
  - Walk, S., Majer, N., Schindler, K., Schiele, B.: `New features and insights for pedestrian detection', 2010 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2010, p. 1030–1037.
58. 58)
  - K. Mikolajczyk , C. Schmid . A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. , 10 , 1615 - 1629
59. 59)
  - Leibe, B., Leonardis, A., Schiele, B.: `Combined object categorization and segmentation with an implicit shape model', ECCV Workshop on Statistical Learning in Computer Vision, 2004, p. 17–32.
60. 60)
  - T. Tuytelaars , K. Mikolajczyk . Local invariant feature detectors: a survey. Found. Trends Comput. Graph. Vis. , 3 , 177 - 280
61. 61)
  - C. Wojek , B. Schiele . A performance evaluation of single and multi-feature people detection. Pattern Recogn. , 82 - 91
62. 62)
  - R.M. Haralick , K. Shanmugam , I. Dinstein . Textural features for image classification. IEEE Trans. Syst. Cybern. , 610 - 621
63. 63)
  - Felzenszwalb, P.F., Girshick, R.B., McAllester, D.: `Cascade object detection with deformable part models', 2010 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2010, p. 2241–2248.
64. 64)
  - Lowe, D.G.: `Object recognition from local scale-invariant features', Int. Conf. on Computer Vision, 1999, 2, p. 1150–1157.
65. 65)
  - Seemann, E., Leibe, B., Schiele, B.: `Multi-aspect detection of articulated objects', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2006, 2, p. 1582–1588.
66. 66)
  - M. Enzweiler , D.M. Gavrila . Monocular pedestrian detection: survey and experiments. IEEE Trans. Pattern Anal. Mach. Intell. , 12 , 2179 - 2195
67. 67)
  - Leibe, B., Schiele, B.: `Interleaved object categorization and segmentation', British Machine Vision Conf., 2003.
68. 68)
  - C. Wöhler , J. Anlauf . An adaptable time-delay neural-network algorithm for image sequence analysis. IEEE Trans. Neural Netw. , 6 , 1531 - 1536
69. 69)
  - Opelt, A., Pinz, A., Zisserman, A.: `A boundary-fragment-model for object detection', The Ninth European Conf. on Computer Vision, 2006, 3952, p. 575.
70. 70)
  - Benfold, B., Reid, I.: `Guiding visual surveillance by tracking human attention', BMVC, 2009.
71. 71)
  - Benfold, B., Reid, I.: `Stable multi-target tracking in real-time surveillance video', Computer Vision and Pattern Recognition, 2011, p. 3457–3464.
72. 72)
  - C. Papageorgiou , T. Poggio . A trainable system for object detection. Int. J. Comput. Vis. , 1 , 15 - 33
73. 73)
  - D. Gerónimo , A.M. López , A.D. Sappa , T. Graf . Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans. Pattern Anal. Mach. Intell. , 7 , 1239 - 1258
74. 74)
  - P.A. Viola , M.J. Jones , D. Snow . Detecting pedestrians using patterns of motion and appearance. Int. J. Comput. Vis. , 2 , 153 - 161
75. 75)
  - Overett, G., Petersson, L., Brewer, N., Andersson, L., Pettersson, N.: `A new pedestrian dataset for supervised learning', IEEE Intelligent Vehicles Symp., 2008, p. 373–378.
76. 76)
  - I.P. Alonso , D.F. Llorca , M.A. Sotelo . Combination of feature extraction methods for svm pedestrian detection. Intell. Trans. Syst. , 2 , 292 - 307
77. 77)
  - Huang, C., Nevatia, R.: `High performance object detection by collaborative learning of joint ranking of granule features', IEEE Conf. on Computer Vision and Pattern Recognition, 2010, p. 41–48.
78. 78)
  - Tuzel, O., Porikli, F., Meer, P.: `Region covariance: a fast descriptor for detection and classification', The Ninth European Conf. on Computer Vision, 2006, 3952, p. 589–000.
79. 79)
  - Hadid, A., Pietikainen, M., Ahonen, T.: `A discriminative feature space for detecting and recognizing faces', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2004, 2.
80. 80)
  - D.G. Lowe . Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis , 2 , 91 - 110
81. 81)
  - Sabzmeydani, P., Mori, G.: `Detecting pedestrians by learning shapelet features', Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 2007, p. 1–8.
82. 82)
  - R.E. Schapire , Y. Singer . Improved boosting algorithms using confidence-rated predictions. Mach. Learn. , 3 , 297 - 336
83. 83)
  - K. Mikolajczyk , C. Schmid . Scale & affine invariant interest point detectors. Int. J. Comput. Vis. , 1 , 63 - 86
84. 84)
  - Comaniciu, D.: `Nonparametric information fusion for motion estimation', IEEE Conf. on Computer Vision and Pattern Recognition, 2003, 1.
85. 85)
  - Dollár, P., Belongie, S., Perona, P.: `The fastest pedestrian detector in the west', BMVC, 2010.
86. 86)
  - Wang, X., Han, T.X., Yan, S.: `An hog-lbp human detector with partial occlusion handling', IEEE 10th Int. Conf. on Computer Vision, 2009.
87. 87)
  - Zhu, Q., Avidan, S., Yeh, M.C., Cheng, K.T.: `Fast human detection using a cascade of histograms of oriented gradients', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2006, 2, p. 1491–1498.
88. 88)
  - Gavrila, D.: `Pedestrian detection from a moving vehicle', European Conf. on Computer Vision, 2000, 1843, p. 37–49.
89. 89)
  - Dollár, P., Wojek, C., Schiele, B., Perona, P.: `Pedestrian detection: a benchmark', IEEE Conf. on Computer Vision and Pattern Recognition, 2009, p. 1–8.
90. 90)
  - T. Ahonen , A. Hadid , M. Pietikäinen . Face description with local binary patterns: application to face recognition. IEEE Trans. Patt. Anal. Mach. Intell. , 12 , 2037 - 2041
91. 91)
  - P.F. Felzenszwalb , R.B. Girshick , D. McAllester , D. Ramanan . Object detection with discriminatively trained part-based models. IEEE Trans. Pattern Anal. Mach. Intell. , 9 , 1627 - 1645
92. 92)
  - Wu, B., Nevatia, R.: `Cluster boosted tree classifier for multi-view, multi-pose object detection', IEEE 11th Int. Conf. on Computer Vision, 2007, p. 1–8.
93. 93)
  - Porikli, F.: `Integral histogram: a fast way to extract histograms in cartesian spaces', IEEE Conf on Computer Vision and Pattern Recognition, 2005, 1, p. 829–000.
94. 94)
  - Jones, M., Viola, P.: `Fast multi-view face detection', IEEE Conf. on Computer Vision and Pattern Recognition, 2003.
95. 95)
  - Lampert, C.H., Blaschko, M.B., Hofmann, T.: `Beyond sliding windows: object localization by efficient subwindow search', Computer Vision and Pattern Recognition, 2008, p. 1–8.
96. 96)
  - Wojek, C., Dorkó, G., Schulz, A., Schiele, B.: `Sliding-windows for rapid object class localization: a parallel technique', Conf. on Computer Vision and Pattern Recognition, 2008, p. 71–81.
97. 97)
  - Leibe, B., Seemann, E., Schiele, B.: `Pedestrian detection in crowded scenes', IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005, 1, p. 878.
98. 98)
  - S. Munder , D.M. Gavrila . An experimental study on pedestrian classification. IEEE Trans. Pattern Anal. Mach. Intell. , 11 , 1863 - 1868
99. 99)
  - Lin, Z., Davis, L.: `A pose-invariant descriptor for human detection and segmentation', ECCV, 2008, p. 423–436.

Backgroundless detection of pedestrians in cluttered conditions based on monocular images: a review

Backgroundless detection of pedestrians in cluttered conditions based on monocular images: a review

Buy article PDF

Buy Knowledge Pack

Thank you

References

Related content