Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon openaccess Fully in tensor computation manner: one-shot dense 3D structured light and beyond

Tensor computation evolves fast towards a prosperous existence in recent years, e.g. PyTorch. An immediate advantage of using tensor computation is that one does not need to implement low-level parallelism to attain efficient computation, which is of simplicity for both research and application development. The authors began with discovering that a simple manoeuvre ‘tensor shift’ could perform neighbourhood manipulation in very efficient parallel manner. Based on ‘tensor shift’, they derive the tensor version of a renowned correspondence search algorithm: semi-global matching (SGM), which they prefix the name as tensor-SGM. To evaluate their idea, they build-up a novel and practical one-shot structured light 3D acquisition system, which yields state-of-art reconstruction results using off-the-shelf hardware. This is the first fully tensorised 3D reconstruction system published to the authors’ best knowledge, and it opens new possibilities. A major one is, in the same tensorised framework, they solved the pattern interfering problem which hinders multi-structured light systems from working together. This part is marked as ‘beyond’ in this study to avoid confusing the readers the spotlight: the fully tensorised 3D structured light framework.

References

    1. 1)
      • 10. Ahmed, N., Natarajan, T., Rao, K.R.: ‘Discrete cosine transform’, IEEE Trans. Comput., 1974, 100, (1), pp. 9093.
    2. 2)
      • 27. Facciolo, G., De Franchis, C., Meinhardt, E.: ‘MGM: A significantly more global matching for stereovision’. British Machine Vision Conf. 2015, Swansea, UK, 2015.
    3. 3)
      • 14. Gupta, M., Nayar, S.K.: ‘Micro phase shifting’. 2012 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA, 2012, pp. 813820.
    4. 4)
      • 26. Bleyer, M., Rhemann, C., Rother, C.: ‘Patchmatch stereo-stereo matching with slanted support windows’. British Machine Vision Conf., Dundee, UK, 2011, vol. 11, pp. 111.
    5. 5)
      • 3. Hirschmuller, H.: ‘Accurate and efficient stereo processing by semi-global matching and mutual information’. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005. CVPR 2005, San Diego, USA, 2005, vol. 2, pp. 807814.
    6. 6)
      • 28. Spangenberg, R., Langner, T., Rojas, R.: ‘Weighted semi-global matching and center-symmetric census transform for robust driver assistance’. Int. Conf. on Computer Analysis of Images and Patterns, York, UK, 2013, pp. 3441.
    7. 7)
      • 32. Pennebaker, W.B., Mitchell, J.L.: ‘JPEG: still image data compression standard’ (Springer Science and Business Media, Berlin, Germany, 1992).
    8. 8)
      • 18. Ulusoy, A.O., Calakli, F., Taubin, G.: ‘One-shot scanning using de bruijn spaced grids’. 2009 IEEE 12th Int. Conf. on Computer Vision Workshops (ICCV Workshops), Kyoto, Japan, 2009, pp. 17861792.
    9. 9)
      • 39. Digne, J., Morel, J.M., Souzani, C.M., et al: ‘Scale space meshing of raw data point sets’, in ‘Computer graphics forum’ (Wiley Online Library, Hoboken, NJ, USA, 2011), vol. 30, pp. 16301642.
    10. 10)
      • 8. Chen, X., Van Gool, L.: ‘Fully in tensor computation manner: a one-shot dense 3D acquisition method using structured light’. 2nd China Symp. on Cognitive Computing and Hybrid Intelligence (CCHI), Xi'an, China, 2019, pp. 2429.
    11. 11)
      • 33. Boykov, Y., Veksler, O., Zabih, R.: ‘Fast approximate energy minimization via graph cuts’, IEEE Trans. Pattern Anal. Mach. Intell., 2001, 23, (11), pp. 12221239.
    12. 12)
      • 31. Žbontar, J., LeCun, Y.: ‘Stereo matching by training a convolutional neural network to compare image patches’, J. Mach. Learn. Res., 2016, 17, (1–32), p. 2.
    13. 13)
      • 35. Moreno, D., Taubin, G.: ‘Simple, accurate, and robust projector-camera calibration’. 2012 Second Int. Conf. on 3D Imaging, Modeling, Processing, Visualization and Transmission (3DIMPVT), Zürich, Switzerland, 2012, pp. 464471.
    14. 14)
      • 5. Chen, X., Van Gool, L.: ‘Watermarking structured light patterns for one-shot, extendable 3D scanning’. Eleventh Int. Conf. on Information Optics and Photonics (CIOP 2019), Xi'an, China, 2019, https://doi.org/10.1117/12.2549975.
    15. 15)
      • 36. Hirschmuller, H., Gehrig, S.: ‘Stereo matching in the presence of sub-pixel calibration errors’. CVPR 2009. IEEE Conf. on Computer Vision and Pattern Recognition, Miami, FL, USA, 2009, pp. 437444.
    16. 16)
      • 17. Kawasaki, H., Furukawa, R., Sagawa, R., et al: ‘Dynamic scene shape reconstruction using a single structured light pattern’. IEEE Conf. on Computer Vision and Pattern Recognition, 2008. CVPR 2008, Anchorage, Alaska, USA, 2008, pp. 18.
    17. 17)
      • 22. Sagawa, R., Furukawa, R., Matsumoto, A., et al: ‘Learning-based feature extraction for active 3d scan with reducing color crosstalk of multiple pattern projections’. 2017 IEEE Int. Conf. on Robotics and Automation (ICRA), Marina Bay Sands, Singapore, 2017, pp. 50745080.
    18. 18)
      • 13. Ryan Fanello, S., Rhemann, C., Tankovich, V., et al: ‘Hyperdepth: learning depth from structured light without matching’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 54415450.
    19. 19)
      • 1. Pytorch: http://pytorch.org, accessed: September 2019.
    20. 20)
      • 30. Michael, M., Salmen, J., Stallkamp, J., et al: ‘Real-time stereo vision: optimizing semi-global matching’. Intelligent Vehicles Symp. (IV), 2013 IEEE, Gold Coast, Australia, 2013, pp. 11971202.
    21. 21)
      • 2. Cambricon: http://www.cambricon.com, accessed: April 2020.
    22. 22)
      • 40. Tomasi, C., Manduchi, R.: ‘Bilateral filtering for gray and color images’. Sixth Int. Conf. on Computer Vision (IEEE Cat. No.98CH36271), Bombay, India, 1998, pp. 839846.
    23. 23)
      • 23. Martinez, M., Stiefelhagen, R.: ‘Kinect unleashed: getting control over high resolution depth maps’. Machine Vision Applications, Kyoto, Japan, 2013, pp. 247250.
    24. 24)
      • 20. Proesmans, M., Van Gool, L., Oosterlinck, A.: ‘One-shot active 3D shape acquisition’. Proc. of the 13th Int. Conf. on Pattern Recognition, 1996, Vienna, Austria, 1996, vol. 3, pp. 336340.
    25. 25)
      • 19. Boyer, K.L., Kak, A.C.: ‘Color-encoded structured light for rapid active ranging’, IEEE Trans. Pattern Anal. Mach. Intell., 1987, PAMI-9, (1), pp. 1428.
    26. 26)
      • 11. Oppenheim, A.V.: ‘Discrete-time signal processing’ (Pearson Higher Ed, USA, 2011, 3rd edn.).
    27. 27)
      • 41. Ito, K., Maeda, K., Fujiwara, I., et al: ‘Time change in strain distribution on the lateral and plantar surfaces of the foot during walking’, Foot. Ankle. Surg., 2016, 2, (22), p. 114.
    28. 28)
      • 16. Petkovic, T., Pribanic, T., Donlic, M., et al: ‘Efficient separation between projected patterns for multiple projector 3d people scanning’. IEEE Int. Conf. on Computer Vision Workshops (ICCV Workshops), Venice, 2017.
    29. 29)
      • 38. Hartley, R.I.: ‘Theory and practice of projective rectification’, Int. J. Comput. Vis., 1999, 35, (2), pp. 115127.
    30. 30)
      • 4. Hartley, R., Zisserman, A.: ‘Multiple view geometry in computer vision’ (Cambridge University Press, Cambridge, UK, 2003).
    31. 31)
      • 6. Moons, T., Van Gool, L., Vergauwen, M.: ‘3d reconstruction from multiple images’. Found. Trends Comput. Graph. Vis., 2009, 4, (4), pp. 287404.
    32. 32)
      • 9. Zabih, R., Woodfill, J.: ‘Non-parametric local transforms for computing visual correspondence’. European Conf. On Computer Vision, Stockholm, Sweden, 1994, pp. 151158.
    33. 33)
      • 24. Sinha, S.N., Scharstein, D., Szeliski, R.: ‘Efficient high-resolution stereo matching using local plane sweeps’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, Ohio, USA, 2014, pp. 15821589.
    34. 34)
      • 34. Kass, M., Witkin, A., Terzopoulos, D.: ‘Snakes: ‘active contour models’, Int. J. Comput. Vis., 1988, 1, (4), pp. 321331.
    35. 35)
      • 12. Vo, M., Narasimhan, S.G., Sheikh, Y.: ‘Texture illumination separation for single-shot structured light reconstruction’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (2), pp. 390404.
    36. 36)
      • 37. Monasse, P.: ‘Quasi-Euclidean epipolar rectification’, Image Process. On Line, 2011, 1, pp. 187199.
    37. 37)
      • 25. Seki, A., Pollefeys, M.: ‘Patch based confidence prediction for dense disparity map’. British Machine Vision Conf., York, UK, 2016.
    38. 38)
      • 15. Couture, V., Martin, N., Roy, S.: ‘Unstructured light scanning to overcome interreflections’. 2011 IEEE Int. Conf. on Computer Vision (ICCV), Barcelona, Spain, 2011, pp. 18951902.
    39. 39)
      • 7. Sagawa, R., Furukawa, R., Kawasaki, H.: ‘Dense 3d reconstruction from high frame-rate video using a static grid pattern’, IEEE Trans. Pattern Anal. Mach. Intell., 2014, 36, (9), pp. 17331747.
    40. 40)
      • 21. Vuylsteke, P., Oosterlinck, A.: ‘Range image acquisition with a single binary-encoded light pattern’, IEEE Trans. Pattern Anal. Mach. Intell., 1990, 12, (2), pp. 148164.
    41. 41)
      • 29. Drory, A., Haubold, C., Avidan, S., et al: ‘Semi-global matching: a principled derivation in terms of message passing’. German Conf. on Pattern Recognition, Münster, Germany, 2014, pp. 4353.
http://iet.metastore.ingenta.com/content/journals/10.1049/ccs.2019.0027
Loading

Related content

content/journals/10.1049/ccs.2019.0027
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address