Learning multi-planar scene models in multi-camera videos

Fei Yin; Sergio A. Velastin; Tim Ellis; Dimitrios Makris

Learning multi-planar scene models in multi-camera videos

View Fulltext

Author(s): Fei Yin ¹ ; Sergio A. Velastin ^{2, 3} ; Tim Ellis ⁴ ; Dimitrios Makris ⁴
- Affiliations: 1: College of Information and Management Science, Henan Agriculture University, Zhengzhou 450002, People's Republic of China;
  2: Department of Informatic Engineering, Universidad de Santiago de Chile, Chile;
  3: Departamento de Informática, Universidad Carlos III de Madrid, Spain;
  4: Digital Imaging Research Centre, School of Computing and Information Systems, Faculty of Science, Engineering and Computing, Kingston University, Penrhyn Road, Kingston upon Thames, Surrey, KT1 2EE, UK
Source: Volume 9, Issue 1, February 2015, p. 25 – 40
DOI: 10.1049/iet-cvi.2013.0261 , Print ISSN 1751-9632, Online ISSN 1751-9640

Received 11/11/2013, Accepted 23/04/2014, Revised 04/04/2014, Published 06/06/2014

Many man-made environments are constructed with multiple levels where people walk, joined by stairs, ramps and overpasses. This study proposes a novel method to learn the geometry of a scene containing more than a single ground plane by tracking pedestrians and combining information from multiple views. The method estimates a scene model with multiple planes by measuring the variation of pedestrian heights across each camera's field of view. It segments the image into separate plane regions, estimating the relative depth and altitude for each image pixel, thus building a three-dimensional reconstruction of the scene. By estimating the multiple planes, the method enables tracking algorithms to follow objects (pedestrians and/or vehicles) that are moving on different ground planes in the scene. The authors also introduce what they believe is the first public dataset with pedestrian traffic on multiple planes to encourage other researchers to compare their work in this field.

References

1. 1)
  - 25. Xu, M., Ellis, T.J.: ‘Partial observation vs. blind tracking through occlusion’. British Machine Vision Conf., BMVA, Cardiff, September 2002, pp. 777–786.
2. 2)
  - 22. Yin, F., Makris, D., Orwell, J., Velastin, S.A.: ‘Learning non-coplanar scene models by exploring the height variation of tracked objects’. ACCV 2010, 2011(LNCS6494), pp. 262–275.
3. 3)
  - 9. Hartley, R., Zisserman, A.: ‘Multiple view geometry in computer vision’ (Cambridge University, Press, Cambridge, UK, 2000).
4. 4)
  - 21. Noceti, N., Balduzzi, L., Odone, F.: ‘What epipolar geometry can do for video-surveillance’. Image Analysis and Processing – ICIAP 2013, Springer Berlin Heidelberg, 2013, pp. 442–451.
5. 5)
  - 18. Rother, D., Patwardhan, K.A., Sapiro, G.: ‘What can casual walkers tell us about a 3D scene?’. IEEE 11th Int. Conf. on Computer Vision, ICCV2007, 2007, pp. 1–8.
6. 6)
  - 29. Chum, O., Matas, J.: ‘Randomized ransac with T(d,d) test’. Proc. of the British Machine Vision Conf., 2002, vol. 2, pp. 448–457.
7. 7)
  - 27. Cleveland, W.S., Devlin, S.J.: ‘Locally-weighted regression: an approach to regression analysis by local fitting’, J. Am. Stat. Assoc., 1988, 83, (403), pp. 596–610 (doi: 10.1080/01621459.1988.10478639).
8. 8)
  - 2. Black, J., Ellis, T.: ‘Multi camera image tracking’. Proc. IEEE Workshop on Performance Evaluation of Tracking and Surveillance, 2001.
9. 9)
  - 28. Lin, L., Zhu, L., Yang, F., Jiang, T.: ‘A novel pixon-representation for image segmentation based on Markov random field’, Image Vis. Comput., 2008, 26, (11), pp. 1507–1514 (doi: 10.1016/j.imavis.2008.04.013).
10. 10)
  - 24. OpenCV (n.d.). Open source computer vision library. http://www.opencv.org/ (accessed: December 2013).
11. 11)
  - 3. Stauffer, C., Tieu, K.: ‘Automated multi-camera planar tracking correspondence modelling’. Proc. of CVPR, 2003, p. 259.
12. 12)
  - 8. Huang, C., Wu, B., Nevatia, R.: ‘Robust object tracking by hierarchical association of detection responses’. ECCV, Marseille, France, 2008, pp. 788–801.
13. 13)
  - 16. Saxena, A., Sun, M., Ng, A.: ‘Make3d: learning 3D scene structure from a single still image’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (5), pp. 824–840 (doi: 10.1109/TPAMI.2008.132).
14. 14)
  - 15. Renno, J.R., Orwell, J., Jones, G.A.: ‘Learning surveillance tracking models for the self-calibrated ground plane’. British Machine Vision Conf., Cardiff, September 2002, pp. 607–616.
15. 15)
  - 12. Wilczkowiak, M., Boyer, E., Sturm, P.: ‘Camera calibration and 3D reconstruction from single images using parallelepipeds’. Proc. Int. Conf. Computer Vision, 2001, pp. 142–148.
16. 16)
  - 16. Krahnstoever, N., Mendonca, P.: ‘Autocalibration from tracks of walking people’. British Machine Vision Conf., Edinburgh, UK, 2006.
17. 17)
  - 1. Madden, C., Cheng, E.D., Piccard, M.: ‘Tracking people across disjoint camera views by an illumination-tolerant appearance representation’, Mach. Vis. Appl., 2007, 18, (3–4), pp. 233–247 (doi: 10.1007/s00138-007-0070-6).
18. 18)
  - S. Khan , M. Shah . Consistent labeling of tracked objects in multiple cameras with overlapping fields of view. IEEE Trans. Pattern Anal. Machine Intell. , 10 , 1355 - 1360
19. 19)
  - 17. Lv, F., Zhao, T., Nevatia, R.: ‘Camera calibration from video of a walking human’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (9), pp. 1513–1518 (doi: 10.1109/TPAMI.2006.178).
20. 20)
  - 20. Breitenstein, M.D., Sommerlade, E., Leibe, B., van Gool, L., Reid, I.: ‘Probabilistic parameter selection for learning scene structure from video’. British Machine Vision Conf. (BMVC'08), September 2008.
21. 21)
  - 19. Fouhey, D., Delaitre, V., Gupta, A., Efros, A.A., Laptev, I., Sivic, J.: ‘People watching: human actions as a cue for single view geometry’. Computer Vision – ECCV 2012, Springer Berlin Heidelberg, 2012, pp. 732–745.
22. 22)
  - 10. Matas, J., Chum, O., Urban, M., Pajdla, T.: ‘Robust wide-baseline stereo from maximally stable extremal regions’, Image Vis. Comput., 2004, 22, (10), pp. 761–767 (doi: 10.1016/j.imavis.2004.02.006).
23. 23)
  - 26. Yin, F., Makris, D., Velastin, S.A., Orwell, J.: ‘Quantitative evaluation of different aspects of motion trackers under various challenges’. Annals of the British Machine Vision Association, (5) British Machine Vision Association, 2010, pp. 1–11.
24. 24)
  - 23. Greenhill, D., Renno, J.R., Orwell, J., Jones, G.A.: ‘Occlusion analysis: learning and utilising depth maps in object tracking, in image and vision computing’. Special Issue on the 15th Annual British Machine Vision Conf., Elsevier Publishing, March 2008, vol. 26, no. 3, pp. 430–44.
25. 25)
  - 14. Hoiem, D., Efros, A.A., Hebert, M.: ‘Putting objects in perspective’, Int. J. Comput. Vis., 2008, 80, (1), pp. 3–15 (doi: 10.1007/s11263-008-0137-5).
26. 26)
  - D.G. Lowe . Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis , 2 , 91 - 110
27. 27)
  - 6. Black, J., Ellis, T., Makris, D.: ‘Wide area surveillance with a multi camera network’ (Intelligent Distributed Surveillance Systems, London, 2004) pp. 21–25.
28. 28)
  - 5. Borg, M., Thirde, D.J., Ferryman, J.M., et al: ‘Automated scene understanding for airport aprons’. The 18th Australian Joint Conf. on Artificial Intelligence (AI05) in Sydney, Australia, December 2005, vol. 3809, pp. 593–603.
29. 29)
  - 7. Black, J., Ellis, T.: ‘Multi camera image tracking’, Image Vis. Comput., 2006, 24, (11), pp. 1256–1267 (doi: 10.1016/j.imavis.2005.06.002).

Login

Not registered yet?

Share

Tools

Login to add to favourites

Key

Learning multi-planar scene models in multi-camera videos

References

Related content