This study presents an adaptive segmentation method for pre-processing input data to the patch-based multi-view stereo algorithm. A specially developed greyscale transformation is applied to the input image data, thus redefining the intensity histogram. The Nelder–Mead simplex method is used to adaptively locate an optimised segmentation threshold point in the modified histogram. The transformed input image is then segmented using the acquired threshold value, into foreground and background data. The segmentation information acquired is applied to the initial feature extraction and the cyclic patch-expansion procedure to constrain the reconstruction to a three-dimensional visibility space that excludes background artefacts. The method is targeted at segmenting out potentially disruptive data and is able to realise a reduction in cumulative error of the reconstruction process and thus improve the final reconstruction. With this method, the authors obtain results that are relatively similar to the original patch-based method, but with reduced time and space complexity.

References

1. 1)
  - 20. Furukawa, Y.: ‘PMVS (Version 2) Documentation’, September 2010. Available at: http://www.di.ens.fr/pmvs/documentation.html. [Accessed 24 November 2015].
2. 2)
  - 5. Schmitt, F., Esteban, C.H.: ‘Silhouette and stereo fusion for 3D object modeling’, J. Comput. Vis. Image Underst., 2004, 96, (3), pp. 267–392.
3. 3)
  - 10. Lhuillier, M., Quan, L.: ‘A quasi-dense approach to surface reconstruction from uncalibrated images’, IEEE Trans. Pattern Anal. Mach. Intell., 2005, 27, (3), pp. 418–433.
4. 4)
  - 24. Sinha, S.N., Pollefeys, M.: ‘Multi-view reconstruction using photo-consistency and exact silhouette constraints: A maximum-flow formulation’. Proc. of the Tenth IEEE Int. Conf. on Computer Vision (ICCV), Washington, DC, USA, 2005, pp. 349–356.
5. 5)
  - 15. Furukawa, Y., Curless, B., Seitz, S.M., et al: ‘Towards internet-scale multiview stereo’. Proc. of IEEE Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2010.
6. 6)
  - 12. Agarwal, S., Snavely, N., Simon, I., et al: ‘Building Rome in a day’. Int. Conf. on Computer Vision, Kyoto, Japan, 2009.
7. 7)
  - 25. Kazhdan, M., Bolitho, M., Hoppe, H.: ‘Poisson surface reconstruction’. Proc. of the fourth Eurographics Symp. on Geometry Processing, Aire-la-Ville, Switzerland, 2006, pp. 61–70.
8. 8)
  - 3. Cipolla, R., Vogiatzis, G., Torr, P.H.S.: ‘Multi-view stereo via volumetric graph-cuts’. Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2005.
9. 9)
  - 7. Goesele, M., Snavely, N., Curless, B., et al: ‘Multi-view stereo for community photo collections’. Int. Conf. on Computer Vision, 2007, pp. 265–270.
10. 10)
  - 9. Furukawa, Y., Ponce, J.: ‘Accurate, dense and robust multi-view stereopsis’. IEEE Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2007, pp. 1–8.
11. 11)
  - 11. Strecha, C., von Hansen, W., Van Gool, L., et al: ‘Dense Multi-View Stereo Evaluation’, 2008. Available at: http://www.cvlabwww.epfl.ch/data/multiview/denseMVS.html. [Accessed 24 November 2015].
12. 12)
  - 22. Hornung, A., Kobbelt, L.: ‘Hierarchical volumetric multi-view stereo reconstruction of manifold surfaces on dual graph embedding’. Proc. of IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR), 2006, pp. 219–231.
13. 13)
  - 2. Seitz, S.M., Curless, B., Diebel, J., et al: ‘Multi-View Stereo Evaluation’, 2009. Available at: http://www.vision.middlebury.edu/mview/. [Accessed 24 November 2015].
14. 14)
  - 16. Shan, Q., Curless, B., Furukawa, Y., et al: ‘Occluding contours for multi-view stereo’. Conf. on Computer Vision and Pattern Recognition, Columbus, OH, USA, 2014, pp. 4002–4009.
15. 15)
  - 18. Weingart, M., Vascan, O.: ‘Image segmentation processing – some techniques and experimental results’. The 4th Int. Symp. on Electrical and Electronic Engineering (ISEEE), Romania, 2013.
16. 16)
  - 4. Vogiatzis, G., Hernández, C., Torr, P.H.S., et al: ‘Multi-view stereo via volumetric graph-cuts and occlusion robust photo-consistency’, IEEE Trans. Pattern Anal. Mach. Intell. (TPAMI), 2007, 29, (12), pp. 2241–2246.
17. 17)
  - 8. Ponce, J., Furukawa, Y.: ‘Accurate, dense, and robust multiview stereopsis’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 32, (8), pp. 1362–1376.
18. 18)
  - 21. Nelder, J., Mead, R.: ‘A simplex method for function minimization’, Comput. J., 1965, 7, pp. 308–313.
19. 19)
  - 26. Lague, D., Brodu, N., Leroux, J.: ‘Accurate 3D comparison of complex topography with terrestrial laser scanner: application to the Rangitikei canyon’, ISPRS J. Photogrammetry Remote Sens., 2013, 82, pp. 10–26.
20. 20)
  - 17. Al-amri, S., Salem, S., Kalyankar, N.V., et al: ‘Image Segmentation by using threshold techniques’. CORR, 2010.
21. 21)
  - 19. Rother, C., Kolmogorov, V., Blake, A.: ‘‘Grab-Cut’: interactive foreground extraction using iterated graph cuts’. ACM Transactions on Graphics (TOG) – Proc. of ACM SIGGRAPH 2004, 2004, vol. 23, no. 3, pp. 309–314.
22. 22)
  - 23. Ponce, J., Furukawa, Y.: ‘Carved visual hulls for image-based modeling’, Int. J. Comput. Vis. (IJCV), 2009, 8, (1), pp. 564–577.
23. 23)
  - 1. Seitz, S.M., Curless, B., Diebel, J., et al: ‘A comparison and evaluation of multi-view stereo reconstruction algorithms’. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, Washington, DC, USA, vol. 1, 2006.
24. 24)
  - 6. Seitz, S.M., Goesele, B.M., Curless, B.: ‘Multi-view stereo revisited’. IEEE Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2006, pp. 2402–2409.
25. 25)
  - 13. Agarwal, S., Furukawa, Y., Snavely, N., et al: ‘Reconstructing Rome’, IEEE Comput., 2010, 43, (6), pp. 40–47.
26. 26)
  - 14. Jinxiang, C., Kang, S.B., Szeliski, R.: ‘Handling occlusions in dense multi-view stereo’. IEEE Computer Vision and Pattern Recognition (CVPR), Washington, DC, USA, 2001, pp. 103–110.

Adaptive segmentation for multi-view stereo

References

Related content