Various computer-vision applications involve estimation of multiple motions from images of dynamic scenes. The exact nature of 3D-object motions and the camera parameters are often not known a priori and therefore, the most general motion model (fundamental matrix) is applied. Although the estimation of fundamental matrix and its use for motion segmentation are established, the conditions for segmentation of different types of motions are largely unaddressed. In this study, we analysed the feasibility of motion segmentation using affine-fundamental matrix, focusing on a scene includes multiple planar-motions, viewed by an uncalibrated camera. We show that the successful segmentation of planar motion depends on several scene and motion parameters. Conditions to guarantee successful segmentation are proposed via extensive experiments using synthetic images. Experiments using real-image data were set up to examine the relevance of those conditions to the scenarios in real applications. The experimental results demonstrate the capability of the proposed conditions to correctly predict the outcome of several segmentation scenarios and show the relevance of those conditions in real applications. In practice, the success of motion segmentation could be predicted from obtainable scene and motion parameters. Therefore these conditions serve as a guideline for practitioners in designing motion-segmentation solutions.

References

1. 1)
  - 4. Ma, Y., Soatto, S., Kosecka, J., Sastry, S.S.: ‘An invitation to 3-D vision: from images to geometric models’ (Springer-Verlag, 2005).
2. 2)
  - P.H.S. Torr , D.W. Murray . The development and comparison of robust methods for estimating the fundamental matrix. Int. J. Comput. Vis. , 3 , 271 - 300
3. 3)
  - 29. Hoseinnezhad, R., Bab-Hadiashar, A.: ‘Consistency of robust estimators in multi-structural visual data segmentation’, Pattern Recognit., 2007, 40, (12), pp. 3677–3690 (doi: 10.1016/j.patcog.2007.05.009).
4. 4)
  - 17. Torr, P.H.S., Szeliski, R., Anandan, P.: ‘An integrated Bayesian approach to layer extraction from image sequences’, IEEE Trans. Pattern Anal. Mach. Intell., 2001, 23, (3), pp. 297–303 (doi: 10.1109/34.910882).
5. 5)
  - 7. Mundy, J.L., Zisserman, A.: ‘Geometric invariance in computer vision’ (MIT Press, Cambridge, MA, USA, 1992).
6. 6)
  - 6. Torr, P.H.S., Zisserman, A., Maybank, S.J.: ‘Robust detection of degenerate configurations while estimating the fundamental matrix’, Comput. Vis. Image Underst., 1998, 71, (3), pp. 312–333 (doi: 10.1006/cviu.1997.0559).
7. 7)
  - 13. Basah, S.N., Hoseinnezhad, R., Bab-Hadiashar, A.: ‘Conditions for segmentation of motion with affine fundamental matrix’. Proc. of the Int. Symp. on Visual Computing ISVC, volume 5875 LNCS, Springer-Verlag, Berlin, Heidelberg, 2009, pp. I: 415–424.
8. 8)
  - 40. Bouguet, J.: Camera calibration toolbox for matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html.
9. 9)
  - R.I. Hartley , P. Sturm . Triangulation. Comput. Vis. Image Underst. , 2 , 146 - 157
10. 10)
  - 20. Vidal, R., Hartley, R.: ‘Motion segmentation with missing data using power factorization and GPCA’. Proc. of the IEEE Computer Society Conf. Computer Vision and Pattern Recognition CVPR, 2004, vol. II, pp. 310–316.
11. 11)
  - 30. Hoseinnezhad, R., Bab-Hadiashar, A., Suter, D.: ‘Finite sample bias of robust estimators in segmentation of closely spaced structures: a comparative study’, J. Math. Imaging Vis., 2010, 37, (1), pp. 66–84 (doi: 10.1007/s10851-010-0193-7).
12. 12)
  - 19. Wolf, L., Shashua, A.: ‘Two-body segmentation from two perspective views’. Proc. of the IEEE Computer Society Conf. on Computer Vision and Pattern Recognition CVPR, vol. I, 2001, pp. 263–270.
13. 13)
  - 3. Faugeras, O., Luong, Q.-T., Papadopoulou, T.: ‘The geometry of multiple images: the laws that govern the formation of images of a scene and some of their applications’ (MIT Press, Cambridge, MA, USA, 2001).
14. 14)
  - 31. Hesami, R., Bab Hadiashar, A., Hoseinnezhad, R.: ‘A novel hierarchical technique for range segmentation of large building exteriors’. Proc. of the Int. Symp. on Visual Computing ISVC, 2007, (LNCS 4842), pp. II: 75–85.
15. 15)
  - 32. Torr, P.H.S., Murray, D.W.: ‘Stochastic motion clustering’. Proc. of the European Conf. on Computer Vision ECCV, Secaucus, NJ, USA, 1994, vol. II, pp. 328–337.
16. 16)
  - 34. Weng, J., Huang, T., Ahuja, N.: ‘Motion and structure from two perspective views: algorithms, error analysis, and error estimation’, IEEE Trans. Pattern Anal. Mach. Intell., 1989, 11, (5), pp. 451–476 (doi: 10.1109/34.24779).
17. 17)
  - 26. Klappstein, J., Stein, F., Franke, U.: ‘Detectability of moving objects using correspondences over two and three frames’. Proc. of the DAGM Conf. Pattern Recognition, Springer-Verlag, Berlin, Heidelberg, 2007, pp. 112–121.
18. 18)
  - D. Zhang , G. Lu . Segmentation of moving objects in image sequence: a review. Circuits, Syst. Signal Process. , 2 , 143 - 183
19. 19)
  - 42. Lowe, D.G.: Sift keypoint detector. http://www.cs.ubc.ca/lowe/keypoints/.
20. 20)
  - 11. Shapiro, L.S., Zisserman, A., Brady, M.: ‘3D motion recovery via affine epipolar geometry’, Int. J. Comput. Vis., 1995, 16, (2), pp. 147–182 (doi: 10.1007/BF01539553).
21. 21)
  - 18. Torr, P.H.S.: ‘Geometric motion segmentation and model selection’, Philos. Trans. R. Soc. A, Math. Phys. Eng. Sci., 1998, 356, (1740), pp. 1321–1340 (doi: 10.1098/rsta.1998.0224).
22. 22)
  - 14. Schindler, K., Suter, D., Wang, H.: ‘A model-selection framework for multibody structure-and-motion of image sequences’, Int. J. Comput. Vis., 2008, 79, (2), pp. 159–177 (doi: 10.1007/s11263-007-0111-7).
23. 23)
  - 38. Bab-Hadiashar, A., Suter, D.: ‘Robust segmentation of visual data using ranked unbiased scale estimate’, Robotica, 1999, 17, (6), pp. 649–660 (doi: 10.1017/S0263574799001812).
24. 24)
  - 8. Xu, G., Zhang, Z.: ‘Epipolar geometry in stereo, motion, and object recognition: a unified approach’ (Kluwer Academic Publishers, Norwell, MA, USA, 1996).
25. 25)
  - S. Basah , A. Bab-Hadiashar , R. Hoseinnezhad . Conditions for motion-background segmentation using fundamental matrix. IET Comput. Vis. , 4 , 189 - 200
26. 26)
  - 5. Schindler, K., Suter, D.: ‘Two-view multibody structure-and-motion with outliers through model selection’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (6), pp. 983–995 (doi: 10.1109/TPAMI.2006.130).
27. 27)
  - 12. N.H-H. Image sequences and datasets from Institut of Algorithm and Cognitive System, Universitat Karlsruhe. http://i21www.ira.uka.de/image_sequences/.
28. 28)
  - 23. Vidal, R., Sastry, S.: ‘Optimal segmentation of dynamic scenes from two perspective views’. Proc. of the IEEE Computer Society Conf. Computer Vision and Pattern Recognition CVPR, 2003, vol. II, pp. 281–286.
29. 29)
  - Z. Zhang . Determining epipolar geometry and its uncertainty: a review. Int. J. Comput. Vis. , 2 , 161 - 195
30. 30)
  - 4. Ma, Y., Soatto, S., Kosecka, J., Sastry, S.S.: ‘An invitation to 3-D vision: from images to geometric models’ (Springer-Verlag, 2005).
31. 31)
  - 37. Evans, M., Hastings, N., Peacock, B.: ‘Statistical distributions’ (Wiley, 2000, 3rd edn.).
32. 32)
  - R. Vidal , Y. Ma , S. Sastry . Generalized principal component analysis (GPCA). IEEE Trans. Pattern Anal. Mach. Intell. , 12 , 1945 - 1959
33. 33)
  - 2. Hartley, R., Zisserman, A.: ‘Multiple view geometry in computer vision’ (Cambridge University Press, Cambridge, UK, 2003, 2nd edn.).
34. 34)
  - 33. Armangu, X., Salvi, J.: ‘Overall view regarding fundamental matrix estimation’, Image Vis. Comput., 2003, 21, pp. 205–220 (doi: 10.1016/S0262-8856(02)00154-3).
35. 35)
  - 25. Vidal, R., Ma, Y.: ‘A unified algebraic approach to 2-D and 3-D motion segmentation and estimation’, J. Math. Imaging Vis., 2006, 25, (3), pp. 403–421 (doi: 10.1007/s10851-006-8286-z).
36. 36)
  - 24. Vidal, R., Ma, Y., Soatto, S., Sastry, S.: ‘Two-view multibody structure from motion’, Int. J. Comput. Vis., 2006, 68, pp. 7–25 (doi: 10.1007/s11263-005-4839-7).
37. 37)
  - W. Hu , T. Tan , L. Wang , S. Maybank . A survey on visual surveillance of object motion and behaviors. IEEE Trans. Syst. Man Cybern. C , 3 , 334 - 352
38. 38)
  - D.G. Lowe . Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis , 2 , 91 - 110
39. 39)
  - 15. Shapiro, L.S., Zisserman, A., Brady, M.: ‘Motion from point matches using affine epipolar geometry’. Proc. of the European Conf. on Computer Vision ECCV, Springer, Secaucus, NJ, USA, 1994, vol. II, pp. 73–84.
40. 40)
  - 21. Vidal, R., Ma, Y., Piazzi, J.: ‘A new GPCA algorithm for clustering subspaces by fitting, differentiating and dividing polynomials’. Proc. of the IEEE Computer Society Conf. on Computer Vision and Pattern Recognition CVPR, 2004, pp. I: 510–517.
41. 41)
  - 16. Torr, P.H.S.: ‘Motion segmentation and outlier detection’. Phd thesis, Department of Engineering Science, University of Oxford, 1995.
42. 42)
  - 28. Basah, S.N., Bab-Hadiashar, A., Hoseinnezhad, R.: ‘Conditions for segmentation of 2D translations of 3D objects’. Proc. of the Int. Conf. on Image Analysis and Processing ICIAP, 2009, (LNCS 5716), Berlin, Heidelberg, pp. 82–91.
43. 43)
  - 36. Weisstein, E.W.: ‘Harmonic addition theorem’. From MathWorld-Wolfram Web Resource http://mathworld.wolfram.com/HarmonicAdditionTheorem.html.

Analysis of planar-motion segmentation using affine fundamental matrix

References

Related content