This study addresses the problem of monocular reconstruction of surfaces that deform isometrically, using points tracked in a single image. To tackle this problem, a flat three-dimensional (3D) shape of the surface and its image are used as template. Such deformations are characterised by certain geometric constraints and to reconstruct the surfaces, these constraints have to be properly exploited. Therefore, the authors propose an algebraic formula that aims at the joint expression of the geometric constraints, namely those based on the differential properties and also those based on the upper-bound model. This expression is, in fact, a unique formulation that results from integrating these two types of constraints, and leads to the intended reconstructions, even when the surface is not strictly isometric. The template shape is used to set the parameters of the expression, which is then optimised (along with the projection equations) by means of a semi-definite programming (SDP) problem. This optimisation enables the estimation of 3D positions of the points on the surface. However, and for implementation purposes, this optimisation is applied separately to patches which, together, make up the whole surface. The experimental results show that the proposed approach improves the results from other methods in terms of accuracy.

References

1. 1)
  - 12. Shen, S., Shi, W., Liu, Y.: ‘Monocular 3-D tracking of inextensible deformable surfaces under L2-norm’, IEEE Trans. Image Process., 2010, 19, pp. 512–521.
2. 2)
  - 4. Dai, Y., Li, H., He, M.: ‘A simple prior-free method for non-rigid structure-from-motion factorization’. CVPR, 2012, pp. 2018–2025.
3. 3)
  - 7. Llado, X., Del Bue, A., Agapito, L.: ‘Non-rigid 3D factorization for projective reconstruction’. BMVC, 2005.
4. 4)
  - 3. Paladini, M., Bue, A.D., Stosic, M., et al: ‘Factorization for non-rigid and articulated structure using metric projections’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, 2009, pp. 2898–2905.
5. 5)
  - 17. Pizarro, D., Bartoli, A., Collins, T.: ‘Isowarp and conwarp: warps that exactly comply with weak-perspective projection of deforming objects’. BMVC, 2013.
6. 6)
  - 5. Aanæs, H., Kahl, F.: ‘Estimation of deformable structure and motion’. Workshop on Vision and Modelling of Dynamic Scenes, ECCV, Denmark, 2002.
7. 7)
  - 9. Salzmann, M., Fua, P.: ‘Reconstructing sharply folding surfaces: a convex formulation’. IEEE Conf. on Computer Vision and Pattern Recognition, 2007.
8. 8)
  - 8. Gay-Bellile, V., Perriollat, M., Bartoli, A., et al: ‘Image registration by combining thin-plate splines with a 3D morphable model’. Int. Conf. on Image Processing, 2006.
9. 9)
  - 15. Salzmann, M., Hartley, R., Fua, P.: ‘Convex optimization for deformable surface 3D tracking’. IEEE Int. Conf. on Computer Vision, 2007.
10. 10)
  - 6. Del-Bue, A., Lladó, X., Agapito, L.: ‘Non-rigid metric shape and motion recovery from uncalibrated images using priors’. IEEE Conf. on Computer Vision and Pattern Recognition, New York, 2006.
11. 11)
  - 1. Bartoli, A., Gérard, Y., Chadebecq, F., et al: ‘On template-based reconstruction from a single view: analytical solutions and proofs of well-posedness for developable, isometric and conformal surfaces’. CVPR, 2012, pp. 2026–2033.
12. 12)
  - 13. Perriollat, M., Hartley, R., Bartoli, A.: ‘Monocular template-based reconstruction of inextensible surfaces’, Int. J. Comput. Vis., 2011, 95, (2), pp. 124–137.
13. 13)
  - 2. Salzmann, M., Urtasun, R., Fua, P.: ‘Local deformation models for monocular 3D shape recovery’. IEEE Conf. on Computer Vision and Pattern Recognition, 2008, pp. 1054–1061.
14. 14)
  - 20. Hosseini, S.J., Araujo, H.: ‘3D estimation of isometric surfaces using a ToF-based approach’. ISVC 2014, 2014.
15. 15)
  - 19. Weinberger, K.Q., Saul, L.K.: ‘Unsupervised learning of image manifolds by semidefinite programming’, Int. J. Comput. Vis., 2006, 70, (1), pp. 77–90.
16. 16)
  - 11. Prasad, M., Zisserman, A., Fitzgibbon, A.W.: ‘Single view reconstruction of curved surfaces’. IEEE Conf. on Computer Vision and Pattern Recognition, 2006, pp. 1345–1354.
17. 17)
  - 10. Gumerov, N., Zandifar, A., Duraiswami, A., et al: ‘Structure of applicable surfaces from single views’. European Conf. on Computer Vision, 2004.
18. 18)
  - 14. Salzmann, M., Moreno-Noguer, F., Lepetit, V., et al: ‘Closed-form solution to non-rigid 3D surface registration’. European Conf. on Computer Vision, 2008, pp. 581–594.
19. 19)
  - 18. Ngo, D.T., Ostlund, J., Fua, P.: ‘Template-based monocular 3D shape recovery using Laplacian meshes TPAMI’.
20. 20)
  - 16. Brunet, F., Bartoli, A., Hartley, R.: ‘Monocular template-based 3D surface reconstruction: convex inextensible and nonconvex isometric methods’, Comput. Vis. Image Underst., 2014, 125, pp. 138–154.

SDP-based approach to monocular reconstruction of inextensible surfaces

References

Related content