http://iet.metastore.ingenta.com
1887

Augmented Lagrangian-based approach for dense three-dimensional structure and motion estimation from binocular image sequences

Augmented Lagrangian-based approach for dense three-dimensional structure and motion estimation from binocular image sequences

For access to this article, please select a purchase option:

Buy article PDF
£12.50
(plus tax if applicable)
Buy Knowledge Pack
10 articles for £75.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Computer Vision — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

In this study, the authors propose a framework for stereo–motion integration for dense depth estimation. They formulate the stereo–motion depth reconstruction problem into a constrained minimisation one. A sequential unconstrained minimisation technique, namely, the augmented Lagrange multiplier (ALM) method has been implemented to address the resulting constrained optimisation problem. ALM has been chosen because of its relative insensitivity to whether the initial design points for a pseudo-objective function are feasible or not. The development of the method and results from solving the stereo–motion integration problem are presented. Although the authors work is not the only one adopting the ALMs framework in the computer vision context, to thier knowledge the presented algorithm is the first to use this mathematical framework in a context of stereo–motion integration. This study describes how the stereo–motion integration problem was cast in a mathematical context and solved using the presented ALM method. Results on benchmark and real visual input data show the validity of the approach.

References

    1. 1)
      • C. Strecha , L.J. Van Gool .
        1. Strecha, C., Van Gool, L.J.: ‘Motion – stereo integration for depth estimation’. ECCV, 2002, no. 2, pp. 170185.
        . ECCV , 2 , 170 - 185
    2. 2)
      • J.A. Worby . (2007)
        2. Worby, J.A.: ‘Multi-resolution graph cuts for stereo-motion estimation’. Master's thesis’, University of Toronto, 2007.
        .
    3. 3)
      • W. Richards .
        3. Richards, W.: ‘Structure from stereo and motion’, J. Opt. Soc. Am., 1985, 2, pp. 343349 (doi: 10.1364/JOSAA.2.000343).
        . J. Opt. Soc. Am. , 343 - 349
    4. 4)
      • A. Waxman , J. Duncan .
        4. Waxman, A., Duncan, J.: ‘Binocular image flows: steps towards stereo-motion fusion’, IEEE Trans. Pattern Anal. Mach. Intell., 1986, 8, (6), pp. 715729 (doi: 10.1109/TPAMI.1986.4767853).
        . IEEE Trans. Pattern Anal. Mach. Intell. , 6 , 715 - 729
    5. 5)
      • L. Li , J. Duncan .
        5. Li, L., Duncan, J.: ‘3-d translational motion and structure from binocular image flows’, IEEE Trans. Pattern Anal. Mach. Intell., 1993, 15, (7), pp. 657667 (doi: 10.1109/34.221167).
        . IEEE Trans. Pattern Anal. Mach. Intell. , 7 , 657 - 667
    6. 6)
      • Z. Zhang , O.D. Faugeras .
        6. Zhang, Z., Faugeras, O.D.: ‘Three-dimensional motion computation and object segmentation in a long sequence of stereo frames’, Int. J. Comput. Vis., 1992, 7, (3), pp. 211241 (doi: 10.1007/BF00126394).
        . Int. J. Comput. Vis. , 3 , 211 - 241
    7. 7)
      • K.J. Hanna , N.E. Okamoto .
        7. Hanna, K.J., Okamoto, N.E.: ‘Combining stereo and motion analysis for direct estimation of scene structure’. ICCV, 1993, pp. 357365.
        . ICCV , 357 - 365
    8. 8)
      • S. Malassiotis , M.G. Strintzis .
        8. Malassiotis, S., Strintzis, M.G.: ‘Model-based joint motion and structure estimation from stereo images’, Comput. Vis. Image Underst., 1997, 65, (1), pp. 7994 (doi: 10.1006/cviu.1996.0481).
        . Comput. Vis. Image Underst. , 1 , 79 - 94
    9. 9)
      • K.N. Kutulakos , S.M. Seitz .
        9. Kutulakos, K.N., Seitz, S.M.: ‘A theory of shape by space carving’, Int. J. Comput. Vis., 2000, 38, (3), pp. 199218 (doi: 10.1023/A:1008191222954).
        . Int. J. Comput. Vis. , 3 , 199 - 218
    10. 10)
      • J. Neumann , Y. Aloimonos .
        10. Neumann, J., Aloimonos, Y.: ‘Spatio-temporal stereo using multiresolution subdivision surfaces’, Int. J. Comput. Vis., 2002, 47, (1–3), pp. 181193 (doi: 10.1023/A:1014597925429).
        . Int. J. Comput. Vis. , 181 - 193
    11. 11)
      • M. Gong .
        11. Gong, M.: ‘Enforcing temporal consistency in real-time stereo estimation’. ECCV, 2006, pp. 564577.
        . ECCV , 564 - 577
    12. 12)
      • M. Isard , J. MacCormick .
        12. Isard, M., MacCormick, J.: ‘Dense motion and disparity estimation via loopy belief propagation’. ACCV, 2006, pp. 3241.
        . ACCV , 32 - 41
    13. 13)
      • G. Sudhir , S. Banerjee , K.K. Biswas , R. Bahl .
        13. Sudhir, G., Banerjee, S., Biswas, K.K., Bahl, R.: ‘Cooperative integration of stereopsis and optic flow computation’, J. Opt. Soc. Am. A, 1995, 12, (12), pp. 25642572 (doi: 10.1364/JOSAA.12.002564).
        . J. Opt. Soc. Am. A , 12 , 2564 - 2572
    14. 14)
      • E.S. Larsen , P. Mordohai , M. Pollefeys , H. Fuchs .
        14. Larsen, E.S., Mordohai, P., Pollefeys, M., Fuchs, H.: ‘Temporally consistent reconstruction from multiple video streams’. ICCV, 2007, pp. 18.
        . ICCV , 1 - 8
    15. 15)
      • C. Strecha , L. Van Gool .
        15. Strecha, C., Van Gool, L.: ‘Pde-based multi-view depth estimation’. First Int. Symp. 3D Data Processing Visualization and Transmission (3DPVT02), 2002, vol. 416.
        . First Int. Symp. 3D Data Processing Visualization and Transmission (3DPVT02)
    16. 16)
      • M. Proesmans , L. van Gool , E. Pauwels , A. Oosterlinck .
        16. Proesmans, M., van Gool, L., Pauwels, E., Oosterlinck, A.: ‘Determination of optical flow and its discontinuities using non-linear diffusion’. ECCV, 1994, pp. 295304.
        . ECCV , 295 - 304
    17. 17)
      • Y. Zhang , C. Kambhamettu .
        17. Zhang, Y., Kambhamettu, C.: ‘On 3-d scene flow and structure recovery from multiview image sequences’, Syst. Man Cybern. B, 2003, 33, (4), pp. 592606 (doi: 10.1109/TSMCB.2003.814284).
        . Syst. Man Cybern. B , 4 , 592 - 606
    18. 18)
      • J.P. Pons , R. Keriven , O. Faugeras .
        18. Pons, J.P., Keriven, R., Faugeras, O.: ‘Modelling dynamic scenes by registering multiview image sequences’. Int. Conf. Computer Vision and Pattern Recognition, 2005, vol. 2, pp. 822827.
        . Int. Conf. Computer Vision and Pattern Recognition , 822 - 827
    19. 19)
      • F. Huguet , F. Devernay .
        19. Huguet, F., Devernay, F.: ‘A variational method for scene flow estimation from stereo sequences’. ICCV, 2007, pp. 17.
        . ICCV , 1 - 7
    20. 20)
      • M. Sizintsev , R. Wildes .
        20. Sizintsev, M., Wildes, R.: ‘Spatiotemporal stereo and scene flow via stequel matching’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (6), pp. 12061219 (doi: 10.1109/TPAMI.2011.202).
        . IEEE Trans. Pattern Anal. Mach. Intell. , 6 , 1206 - 1219
    21. 21)
      • L. Valgaerts , A. Bruhn , H. Zimmer , J. Weickert , C. Stoll , C. Theobalt .
        21. Valgaerts, L., Bruhn, A., Zimmer, H., Weickert, J., Stoll, C., Theobalt, C.: ‘Joint estimation of motion, structure and geometry from stereo sequences’. ECCV, 2010, (2010).
        . ECCV
    22. 22)
      • G. De Cubber .
        22. De Cubber, G.: ‘Variational methods for dense depth reconstruction from monocular and binocular sequences’. PhD thesis, Vrije Universiteit Brussel, March 2010.
        .
    23. 23)
      • A. Del Bue , J. Xavier , L. Agapito , M. Paladini .
        23. Del Bue, A., Xavier, J., Agapito, L., Paladini, M.: ‘Bilinear modeling via augmented lagrange multipliers (balm)’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (8), pp. 14961508 (doi: 10.1109/TPAMI.2011.238).
        . IEEE Trans. Pattern Anal. Mach. Intell. , 8 , 1496 - 1508
    24. 24)
      • J. Nocedal , S.J. Wright . (1999)
        24. Nocedal, J., Wright, S.J.: ‘Numerical optimizationSpringer series in operations research. (Springer, 1999, 2nd edn.).
        .
    25. 25)
      • G. De Cubber , H. Sahli .
        25. De Cubber, G., Sahli, H.: ‘Partial differential equation-based dense 3d structure and motion estimation from monocular image sequences’, IET Comput. Vis., 2012, 6, (3), pp. 174185 (doi: 10.1049/iet-cvi.2011.0174).
        . IET Comput. Vis. , 3 , 174 - 185
    26. 26)
      • P. Fua .
        26. Fua, P.: ‘Combining stereo and monocular information to compute dense depth maps that preserve depth discontinuities’. 12th Int. Joint Conf. Artificial Intelligence, 1991, pp. 12921298.
        . 12th Int. Joint Conf. Artificial Intelligence , 1292 - 1298
    27. 27)
      • D. Murray , J.J. Little .
        27. Murray, D., Little, J.J.: ‘Using real-time stereo vision for mobile robot navigation’, Auton. Robots, 2000, 8, (2), pp. 161171 (doi: 10.1023/A:1008987612352).
        . Auton. Robots , 2 , 161 - 171
    28. 28)
      • D.P. Bertsekas . (1996)
        28. Bertsekas, D.P.: ‘Constrained optimization and lagrange multiplier methods’ (Athena Scientific, 1996).
        .
    29. 29)
      • M.J.D. Powell . (1969)
        29. Powell, M.J.D.: ‘OptimizationA method of nonlinear constraints in minimization problems, (Academic Press, London, 1969).
        .
    30. 30)
      • M.R. Hestenes .
        30. Hestenes, M.R.: ‘Multipler and gradient methods’, J. Optim. Theory Appl., 1969, 4, pp. 303320 (doi: 10.1007/BF00927673).
        . J. Optim. Theory Appl. , 303 - 320
    31. 31)
      • H. Nagel , W. Enkelmann .
        31. Nagel, H., Enkelmann, W.: ‘An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences’, IEEE Trans. Pattern Anal. Mach. Intell., 1986, 8, (5), pp. 565593 (doi: 10.1109/TPAMI.1986.4767833).
        . IEEE Trans. Pattern Anal. Mach. Intell. , 5 , 565 - 593
    32. 32)
      • R. Keys .
        32. Keys, R.: ‘Cubic convolution interpolation for digital image processing’, IEEE Trans. Acoust. Speech Signal Process., 1981, 29, (6), pp. 11531160 (doi: 10.1109/TASSP.1981.1163711).
        . IEEE Trans. Acoust. Speech Signal Process. , 6 , 1153 - 1160
    33. 33)
      • R.P. Brent . (1973)
        33. Brent, R.P.: ‘Algorithms for minimization without derivatives’ (Prentice-Hall, Englewood Cliffs, NJ, 1973).
        .
    34. 34)
      • G.E. Forsythe , M.A. Malcolm , C.B. Moler . (1976)
        34. Forsythe, G.E., Malcolm, M.A., Moler, C.B.: ‘Computer methods for mathematical computations’ (Prentice-Hall, 1976).
        .
    35. 35)
      • P.H.S. Torr .
        35. Torr, P.H.S.: ‘Bayesian model estimation and selection for epipolar geometry and generic manifold fitting’, Int. J. Comput. Vis., 2002, 50, (1), pp. 3561 (doi: 10.1023/A:1020224303087).
        . Int. J. Comput. Vis. , 1 , 35 - 61
    36. 36)
      • D. Scharstein , R. Szeliski .
        36. Scharstein, D., Szeliski, R.: ‘A taxonomy and evaluation of dense two-frame stereo correspondence algorithms’, Int. J. Comput. Vis., 2002, 47, (1–3), pp. 742 (doi: 10.1023/A:1014573219977).
        . Int. J. Comput. Vis. , 7 - 42
    37. 37)
      • D. Scharstein , R. Szeliski .
        37. Scharstein, D., Szeliski, R.: ‘High-accuracy stereo depth maps using structured light’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR 2003), Madison, WI, USA, 2003, vol. 1, pp. 195202.
        . IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR 2003) , 195 - 202
    38. 38)
      • P.F. Felzenszwalb , D.P. Huttenlocher .
        38. Felzenszwalb, P.F., Huttenlocher, D.P.: ‘Efficient belief propagation for early vision’, Int. J. Comput. Vis., 2006, 70, (1), pp. 126 (doi: 10.1007/s11263-006-7899-4).
        . Int. J. Comput. Vis. , 1 , 126
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2013.0017
Loading

Related content

content/journals/10.1049/iet-cvi.2013.0017
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address