access icon openaccess Multiuser virtual reality environment for visualising neuroimaging data

The recent advent of high-performance consumer virtual reality (VR) systems has opened new possibilities for immersive visualisation of numerous types of data. Medical imaging has long made use of advanced visualisation techniques, and VR offers exciting new opportunities for data exploration. The author presents a new framework for interacting with neuroimaging data, including MRI volumes, neuroanatomical surface models, diffusion tensors, and streamline tractography, as well as text-based annotations. The system was developed for the HTC Vive using C++, OpenGL, and the OpenVR software development kit. The author developed custom GLSL shaders for each type of data to provide high-performance real-time rendering suitable for use in a VR environment. These are integrated with an interface that enables the user to manipulate the scene through the Vive controllers and perform operations such as volume slicing, fibre track selection, and structural queries. The software can read data generated by existing automated brain MRI analysis packages, enabling the rapid development of subject-specific visualisations of multimodal data or annotated atlases. The system can also support multiple simultaneous users, placing them in the same virtual space to interact with each other while visualising the same datasets, opening new possibilities for teaching and for collaborative exploration of neuroimaging data.

Inspec keywords: brain; neurophysiology; biomedical MRI; rendering (computer graphics); medical image processing; data visualisation; virtual reality

Other keywords: virtual space; medical imaging; HTC Vive; Vive controllers; automated brain MRI analysis packages; streamline tractography; text-based annotations; high-performance consumer virtual reality systems; diffusion tensors; MRI volumes; multiuser virtual reality environment; neuroimaging data; neuroanatomical surface models; fibre track selection; OpenVR software development kit; OpenGL

Subjects: Biophysics of neurophysiological processes; Medical magnetic resonance imaging and spectroscopy; Patient diagnostic methods and instrumentation; Biomedical magnetic resonance imaging and spectroscopy; Virtual reality; Biology and medical computing; Optical, image and video signal processing; Computer vision and image processing techniques

References

    1. 1)
    2. 2)
    3. 3)
      • 9. Joshi, A.A., Shattuck, D.W., Leahy, R.M.: ‘A method for automated cortical surface registration and labeling’, in Dawant, B.M., Christensen, G.E., Fitzpatrick, J.M., Rueckert, D., (Eds.): ‘Biomedical image registration’ (Springer Berlin Heidelberg, Berlin, Heidelberg, 2012), pp. 180189.
    4. 4)
    5. 5)
    6. 6)
      • 21. Toro, R.: ‘Braincatalogue’, 2018. Available from: https://braincatalogue.org/Bottlenose_dolphin, accessed June 10, 2018.
    7. 7)
      • 6. Martin, K.: ‘Using virtual reality devices with VTK’. Available at: https://blog.kitware.com/using-virtual-reality-devices-with-vtk/, accessed: 2018-08-05.
    8. 8)
    9. 9)
      • 7. Arikatla, S., Fillion Robin, J.C., Paniagua, B., et al: ‘Bringing virtual reality to 3D Slicer’. Available at: https://blog.kitware.com/slicervirtualreality/, accessed: 2018-08-05.
    10. 10)
      • 20. Swan, J.E., Yagel, R.: ‘Slice-based volume rendering’ (Ohio State University, Columbus, Ohio, USA, 1993). OSU-ACCAD-1/93-TR1.
    11. 11)
    12. 12)
      • 4. Duncan, D., Garner, R., Zrantchev, I., et al: ‘Using virtual reality to improve performance and user experience in manual correction of MRI segmentation errors by non-experts’, J. Digit. Imaging, 2018, pp. 18, doi: 10.1007/s10278-018-0108-5.
    13. 13)
      • 16. Wang, R., Benner, T., Sorensen, A.G., et al: ‘Diffusion toolkit: a software package for diffusion imaging data processing and tractography’. Proc. Int. Soc. Mag. Reson. Med., Berlin, Germany, 2007, vol. 15, p. 3720.
    14. 14)
      • 3. Ard, T., Krum, D.M., Phan, T., et al: ‘NIVR: neuro imaging in virtual reality’. Proc. IEEE Virtual Reality (VR), Los Angeles, California, USA, 2017, pp. 465466.
    15. 15)
    16. 16)
    17. 17)
    18. 18)
    19. 19)
    20. 20)
      • 19. Rivière, D., Geffroy, D., Denghien, I., et al: ‘BrainVISA: an extensible software environment for sharing multimodal neuroimaging data and processing tools’. Proc. 15th HBM, San Francisco, California, USA, 2009.
    21. 21)
      • 2. Chen, B., Moreland, J., Zhang, J.: ‘Human brain functional MRI and DTI visualization with virtual reality’, Quant. Imaging. Med. Surg., 2011, 1, pp. 1116.
    22. 22)
    23. 23)
    24. 24)
      • 1. Zhang, S., Demiralp, Ç., DaSilva, M., et al: ‘Toward application of virtual reality to visualization of DT-MRI volumes’. Proc. of the 4th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, MICCAI ‘01, Berlin, Heidelberg, 2001, pp. 14061408.
    25. 25)
    26. 26)
    27. 27)
http://iet.metastore.ingenta.com/content/journals/10.1049/htl.2018.5077
Loading

Related content

content/journals/10.1049/htl.2018.5077
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading