This is an open access article published by the IET under the Creative Commons Attribution-NonCommercial-NoDerivs License (http://creativecommons.org/licenses/by-nc-nd/3.0/)
The recent advent of high-performance consumer virtual reality (VR) systems has opened new possibilities for immersive visualisation of numerous types of data. Medical imaging has long made use of advanced visualisation techniques, and VR offers exciting new opportunities for data exploration. The author presents a new framework for interacting with neuroimaging data, including MRI volumes, neuroanatomical surface models, diffusion tensors, and streamline tractography, as well as text-based annotations. The system was developed for the HTC Vive using C++, OpenGL, and the OpenVR software development kit. The author developed custom GLSL shaders for each type of data to provide high-performance real-time rendering suitable for use in a VR environment. These are integrated with an interface that enables the user to manipulate the scene through the Vive controllers and perform operations such as volume slicing, fibre track selection, and structural queries. The software can read data generated by existing automated brain MRI analysis packages, enabling the rapid development of subject-specific visualisations of multimodal data or annotated atlases. The system can also support multiple simultaneous users, placing them in the same virtual space to interact with each other while visualising the same datasets, opening new possibilities for teaching and for collaborative exploration of neuroimaging data.
References
-
-
1)
-
14. MacKenzie Graham, A., Lee, E.F., Dinov, I.D., et al: ‘A multimodal, multidimensional atlas of the C57BL/6J mouse brain’, J. Anat., 2004, 204, pp. 93–102 (doi: 10.1111/j.1469-7580.2004.00264.x).
-
2)
-
27. Garyfallidis, E., Brett, M., Amirbekian, B., et al: ‘Dipy, a library for the analysis of diffusion MRI data’, Front. Neuroinform., 2014, 8, p. 8 (doi: 10.3389/fninf.2014.00008).
-
3)
-
9. Joshi, A.A., Shattuck, D.W., Leahy, R.M.: ‘A method for automated cortical surface registration and labeling’, in Dawant, B.M., Christensen, G.E., Fitzpatrick, J.M., Rueckert, D., (Eds.): ‘Biomedical image registration’ (Springer Berlin Heidelberg, Berlin, Heidelberg, 2012), pp. 180–189.
-
4)
-
22. Le Bihan, D., Johansen Berg, H.: ‘Diffusion MRI at 25: exploring brain tissue structure and function’, NeuroImage, 2012, 61, pp. 324–341 (doi: 10.1016/j.neuroimage.2011.11.006).
-
5)
-
15. Shattuck, D.W., Mirza, M., Adisetiyo, V., et al: ‘Construction of a 3D probabilistic atlas of human cortical structures’, NeuroImage, 2008, 39, pp. 1064–1080 (doi: 10.1016/j.neuroimage.2007.09.031).
-
6)
-
21. Toro, R.: ‘Braincatalogue’, 2018. .
-
7)
-
6. Martin, K.: ‘Using virtual reality devices with VTK’. .
-
8)
-
10. Haldar, J.P., Leahy, R.M.: ‘Linear transforms for Fourier data on the sphere: application to high angular resolution diffusion MRI of the brain’, NeuroImage, 2013, 71, pp. 233–247 (doi: 10.1016/j.neuroimage.2013.01.022).
-
9)
-
7. Arikatla, S., Fillion Robin, J.C., Paniagua, B., et al: ‘Bringing virtual reality to 3D Slicer’. .
-
10)
-
20. Swan, J.E., Yagel, R.: ‘Slice-based volume rendering’ (Ohio State University, Columbus, Ohio, USA, 1993). .
-
11)
-
12. Habibi, A., Ilari, B., Crimi, K., et al: ‘An equal start: absence of group differences in cognitive, social, and neural measures prior to music or sports training in children’, Front. Hum. Neurosci., 2014, 8, p. 690 (doi: 10.3389/fnhum.2014.00690).
-
12)
-
4. Duncan, D., Garner, R., Zrantchev, I., et al: ‘Using virtual reality to improve performance and user experience in manual correction of MRI segmentation errors by non-experts’, J. Digit. Imaging, 2018, pp. 1–8, .
-
13)
-
16. Wang, R., Benner, T., Sorensen, A.G., et al: ‘Diffusion toolkit: a software package for diffusion imaging data processing and tractography’. Proc. Int. Soc. Mag. Reson. Med., Berlin, Germany, 2007, , p. 3720.
-
14)
-
3. Ard, T., Krum, D.M., Phan, T., et al: ‘NIVR: neuro imaging in virtual reality’. Proc. IEEE Virtual Reality (VR), Los Angeles, California, USA, 2017, pp. 465–466.
-
15)
-
18. Fischl, B.: ‘Freesurfer’, NeuroImage, 2012, 62, pp. 774–781 (doi: 10.1016/j.neuroimage.2012.01.021).
-
16)
-
13. Phillips, O.R., Joshi, S.H., Squitieri, F., et al: ‘Major superficial white matter abnormalities in Huntington's disease’, Front. Neurosci., 2016, 10, p. 197 (doi: 10.3389/fnins.2016.00197).
-
17)
-
5. Egger, J., Gall, M., Wallner, J., et al: ‘HTC vive MeVisLab integration via OpenVR for medical applications’, PLoS One, 2017, 12, p. e0173972 (doi: 10.1371/journal.pone.0173972).
-
18)
-
28. Van Essen, D.C., Ugurbil, K., Auerbach, E., et al: ‘The human connectome project: a data acquisition perspective’, NeuroImage, 2012, 62, pp. 2222–2231 (doi: 10.1016/j.neuroimage.2012.02.018).
-
19)
-
11. Bhushan, C., Haldar, J.P., Choi, S., et al: ‘Coregistration and distortion correction of diffusion and anatomical images based on inverse contrast normalization’, NeuroImage, 2015, 115, pp. 269–280 (doi: 10.1016/j.neuroimage.2015.03.050).
-
20)
-
19. Rivière, D., Geffroy, D., Denghien, I., et al: ‘BrainVISA: an extensible software environment for sharing multimodal neuroimaging data and processing tools’. Proc. 15th HBM, San Francisco, California, USA, 2009.
-
21)
-
2. Chen, B., Moreland, J., Zhang, J.: ‘Human brain functional MRI and DTI visualization with virtual reality’, Quant. Imaging. Med. Surg., 2011, 1, pp. 11–16.
-
22)
-
26. Yeh, F.C., Verstynen, T.D., Wang, Y., et al: ‘Deterministic diffusion fiber tracking improved by quantitative anisotropy’, PLoS One, 2013, 8, p. e80713 (doi: 10.1371/journal.pone.0080713).
-
23)
-
23. Basser, P.J., Mattiello, J., LeBihan, D.: ‘MR diffusion tensor spectroscopy and imaging’, Biophys. J., 1994, 66, pp. 259–267 (doi: 10.1016/S0006-3495(94)80775-1).
-
24)
-
1. Zhang, S., Demiralp, Ç., DaSilva, M., et al: ‘Toward application of virtual reality to visualization of DT-MRI volumes’. Proc. of the 4th Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, MICCAI ‘01, Berlin, Heidelberg, 2001, pp. 1406–1408.
-
25)
-
25. Behrens, T.E.J., Johansen Berg, H., Woolrich, M.W., et al: ‘Non-invasive mapping of connections between human thalamus and cortex using diffusion imaging’, Nat. Neurosci., 2003, 6, pp. 750–757 (doi: 10.1038/nn1075).
-
26)
-
8. Shattuck, D.W., Leahy, R.M.: ‘BrainSuite: an automated cortical surface identification tool’, Med. Image Anal., 2002, 6, pp. 129–142 (doi: 10.1016/S1361-8415(02)00054-3).
-
27)
-
17. Norton, I., Essayed, W.I., Zhang, F., et al: ‘Slicerdmri: open source diffusion MRI software for brain cancer research’, Cancer Res., 2017, 77, pp. e101–e103 (doi: 10.1158/0008-5472.CAN-17-0332).
http://iet.metastore.ingenta.com/content/journals/10.1049/htl.2018.5077
Related content
content/journals/10.1049/htl.2018.5077
pub_keyword,iet_inspecKeyword,pub_concept
6
6