Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Augmented Lagrangian-based approach for dense three-dimensional structure and motion estimation from binocular image sequences

  • Author(s): Geert De Cubber 1, 2  and  Hichem Sahli 1, 3
    • View affiliations
    • Affiliations: 1: Electronics and Information Processing (ETRO), Vrije Universiteit Brussel, Brussels 1040, Belgium;
      2: Mechanical Engineering, Royal Military Academy of Belgium, Brussels 1000, Belgium;
      3: Interuniversity Microelectronics Centre – IMEC, Heverlee 3001, Belgium
  • Source: Volume 8, Issue 2, April 2014, p. 98 – 109
    DOI:  10.1049/iet-cvi.2013.0017 , Print ISSN 1751-9632, Online ISSN 1751-9640
© The Institution of Engineering and Technology
Received 24/09/2012, Accepted 16/07/2013, Revised 07/06/2013, Published 18/09/2013

In this study, the authors propose a framework for stereo–motion integration for dense depth estimation. They formulate the stereo–motion depth reconstruction problem into a constrained minimisation one. A sequential unconstrained minimisation technique, namely, the augmented Lagrange multiplier (ALM) method has been implemented to address the resulting constrained optimisation problem. ALM has been chosen because of its relative insensitivity to whether the initial design points for a pseudo-objective function are feasible or not. The development of the method and results from solving the stereo–motion integration problem are presented. Although the authors work is not the only one adopting the ALMs framework in the computer vision context, to thier knowledge the presented algorithm is the first to use this mathematical framework in a context of stereo–motion integration. This study describes how the stereo–motion integration problem was cast in a mathematical context and solved using the presented ALM method. Results on benchmark and real visual input data show the validity of the approach.

Inspec keywords: stereo image processing; image reconstruction; computer vision; minimisation; motion estimation; mathematical analysis; image sequences; constraint handling

Other keywords: pseudoobjective function; ALM; motion estimation; computer vision context; constrained minimisation; augmented Lagrange multiplier method; stereo-motion depth reconstruction problem; mathematical framework; constrained optimisation problem; binocular image sequence; sequential unconstrained minimisation technique; dense depth estimation; stereo-motion integration problem; dense three-dimensional structure

Subjects: Optimisation techniques; Computer vision and image processing techniques; Optical, image and video signal processing; Optimisation techniques; Mathematical analysis; Mathematical analysis

References

    1. 1)
    2. 2)
    3. 3)
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
    9. 9)
    10. 10)
    11. 11)
    12. 12)
    13. 13)
    14. 14)
    15. 15)
    16. 16)
    17. 17)
    18. 18)
    19. 19)
    20. 20)
      • 4. Waxman, A., Duncan, J.: ‘Binocular image flows: steps towards stereo-motion fusion’, IEEE Trans. Pattern Anal. Mach. Intell., 1986, 8, (6), pp. 715–729 (doi: 10.1109/TPAMI.1986.4767853).
    21. 21)
      • 31. Nagel, H., Enkelmann, W.: ‘An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences’, IEEE Trans. Pattern Anal. Mach. Intell., 1986, 8, (5), pp. 565–593 (doi: 10.1109/TPAMI.1986.4767833).
    22. 22)
      • 19. Huguet, F., Devernay, F.: ‘A variational method for scene flow estimation from stereo sequences’. ICCV, 2007, pp. 1–7.
    23. 23)
      • 26. Fua, P.: ‘Combining stereo and monocular information to compute dense depth maps that preserve depth discontinuities’. 12th Int. Joint Conf. Artificial Intelligence, 1991, pp. 1292–1298.
    24. 24)
      • 29. Powell, M.J.D.: ‘Optimization’ A method of nonlinear constraints in minimization problems, (Academic Press, London, 1969).
    25. 25)
      • 14. Larsen, E.S., Mordohai, P., Pollefeys, M., Fuchs, H.: ‘Temporally consistent reconstruction from multiple video streams’. ICCV, 2007, pp. 1–8.
    26. 26)
      • 7. Hanna, K.J., Okamoto, N.E.: ‘Combining stereo and motion analysis for direct estimation of scene structure’. ICCV, 1993, pp. 357–365.
    27. 27)
      • 23. Del Bue, A., Xavier, J., Agapito, L., Paladini, M.: ‘Bilinear modeling via augmented lagrange multipliers (balm)’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (8), pp. 1496–1508 (doi: 10.1109/TPAMI.2011.238).
    28. 28)
      • 35. Torr, P.H.S.: ‘Bayesian model estimation and selection for epipolar geometry and generic manifold fitting’, Int. J. Comput. Vis., 2002, 50, (1), pp. 35–61 (doi: 10.1023/A:1020224303087).
    29. 29)
      • 20. Sizintsev, M., Wildes, R.: ‘Spatiotemporal stereo and scene flow via stequel matching’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (6), pp. 1206–1219 (doi: 10.1109/TPAMI.2011.202).
    30. 30)
      • 17. Zhang, Y., Kambhamettu, C.: ‘On 3-d scene flow and structure recovery from multiview image sequences’, Syst. Man Cybern. B, 2003, 33, (4), pp. 592–606 (doi: 10.1109/TSMCB.2003.814284).
    31. 31)
      • 30. Hestenes, M.R.: ‘Multipler and gradient methods’, J. Optim. Theory Appl., 1969, 4, pp. 303–320 (doi: 10.1007/BF00927673).
    32. 32)
      • 33. Brent, R.P.: ‘Algorithms for minimization without derivatives’ (Prentice-Hall, Englewood Cliffs, NJ, 1973).
    33. 33)
      • 8. Malassiotis, S., Strintzis, M.G.: ‘Model-based joint motion and structure estimation from stereo images’, Comput. Vis. Image Underst., 1997, 65, (1), pp. 79–94 (doi: 10.1006/cviu.1996.0481).
    34. 34)
      • 25. De Cubber, G., Sahli, H.: ‘Partial differential equation-based dense 3d structure and motion estimation from monocular image sequences’, IET Comput. Vis., 2012, 6, (3), pp. 174–185 (doi: 10.1049/iet-cvi.2011.0174).
    35. 35)
      • 21. Valgaerts, L., Bruhn, A., Zimmer, H., Weickert, J., Stoll, C., Theobalt, C.: ‘Joint estimation of motion, structure and geometry from stereo sequences’. ECCV, 2010, (2010).
    36. 36)
      • 12. Isard, M., MacCormick, J.: ‘Dense motion and disparity estimation via loopy belief propagation’. ACCV, 2006, pp. 32–41.
    37. 37)
      • 16. Proesmans, M., van Gool, L., Pauwels, E., Oosterlinck, A.: ‘Determination of optical flow and its discontinuities using non-linear diffusion’. ECCV, 1994, pp. 295–304.
    38. 38)
      • 3. Richards, W.: ‘Structure from stereo and motion’, J. Opt. Soc. Am., 1985, 2, pp. 343–349 (doi: 10.1364/JOSAA.2.000343).
    39. 39)
      • 32. Keys, R.: ‘Cubic convolution interpolation for digital image processing’, IEEE Trans. Acoust. Speech Signal Process., 1981, 29, (6), pp. 1153–1160 (doi: 10.1109/TASSP.1981.1163711).
    40. 40)
      • 38. Felzenszwalb, P.F., Huttenlocher, D.P.: ‘Efficient belief propagation for early vision’, Int. J. Comput. Vis., 2006, 70, (1), pp. 126 (doi: 10.1007/s11263-006-7899-4).
    41. 41)
      • 10. Neumann, J., Aloimonos, Y.: ‘Spatio-temporal stereo using multiresolution subdivision surfaces’, Int. J. Comput. Vis., 2002, 47, (1–3), pp. 181–193 (doi: 10.1023/A:1014597925429).
    42. 42)
      • 34. Forsythe, G.E., Malcolm, M.A., Moler, C.B.: ‘Computer methods for mathematical computations’ (Prentice-Hall, 1976).
    43. 43)
      • 13. Sudhir, G., Banerjee, S., Biswas, K.K., Bahl, R.: ‘Cooperative integration of stereopsis and optic flow computation’, J. Opt. Soc. Am. A, 1995, 12, (12), pp. 2564–2572 (doi: 10.1364/JOSAA.12.002564).
    44. 44)
      • 28. Bertsekas, D.P.: ‘Constrained optimization and lagrange multiplier methods’ (Athena Scientific, 1996).
    45. 45)
      • 1. Strecha, C., Van Gool, L.J.: ‘Motion – stereo integration for depth estimation’. ECCV, 2002, no. 2, pp. 170–185.
    46. 46)
      • 2. Worby, J.A.: ‘Multi-resolution graph cuts for stereo-motion estimation’. Master's thesis’, University of Toronto, 2007.
    47. 47)
      • 11. Gong, M.: ‘Enforcing temporal consistency in real-time stereo estimation’. ECCV, 2006, pp. 564–577.
    48. 48)
      • 27. Murray, D., Little, J.J.: ‘Using real-time stereo vision for mobile robot navigation’, Auton. Robots, 2000, 8, (2), pp. 161–171 (doi: 10.1023/A:1008987612352).
    49. 49)
      • 6. Zhang, Z., Faugeras, O.D.: ‘Three-dimensional motion computation and object segmentation in a long sequence of stereo frames’, Int. J. Comput. Vis., 1992, 7, (3), pp. 211–241 (doi: 10.1007/BF00126394).
    50. 50)
      • 37. Scharstein, D., Szeliski, R.: ‘High-accuracy stereo depth maps using structured light’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR 2003), Madison, WI, USA, 2003, vol. 1, pp. 195–202.
    51. 51)
      • 9. Kutulakos, K.N., Seitz, S.M.: ‘A theory of shape by space carving’, Int. J. Comput. Vis., 2000, 38, (3), pp. 199–218 (doi: 10.1023/A:1008191222954).
    52. 52)
      • 22. De Cubber, G.: ‘Variational methods for dense depth reconstruction from monocular and binocular sequences’. PhD thesis, Vrije Universiteit Brussel, March 2010.
    53. 53)
      • 5. Li, L., Duncan, J.: ‘3-d translational motion and structure from binocular image flows’, IEEE Trans. Pattern Anal. Mach. Intell., 1993, 15, (7), pp. 657–667 (doi: 10.1109/34.221167).
    54. 54)
      • 36. Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two-frame stereo correspondence algorithms’, Int. J. Comput. Vis., 2002, 47, (1–3), pp. 7–42 (doi: 10.1023/A:1014573219977).
    55. 55)
      • 24. Nocedal, J., Wright, S.J.: ‘Numerical optimization’ Springer series in operations research. (Springer, 1999, 2nd edn.).
    56. 56)
      • 18. Pons, J.P., Keriven, R., Faugeras, O.: ‘Modelling dynamic scenes by registering multiview image sequences’. Int. Conf. Computer Vision and Pattern Recognition, 2005, vol. 2, pp. 822–827.
    57. 57)
      • 15. Strecha, C., Van Gool, L.: ‘Pde-based multi-view depth estimation’. First Int. Symp. 3D Data Processing Visualization and Transmission (3DPVT02), 2002, vol. 416.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2013.0017
Loading

Related content

content/journals/10.1049/iet-cvi.2013.0017
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address