Gesture-based registration correction using a mobile augmented reality image-guided neurosurgery system
Abstract
In image-guided neurosurgery, a registration between the patient and their pre-operative images and the tracking of surgical tools enables GPS-like guidance to the surgeon. However, factors such as brainshift, image distortion, and registration error cause the patient-to-image alignment accuracy to degrade throughout the surgical procedure no longer providing accurate guidance. The authors present a gesture-based method for manual registration correction to extend the usage of augmented reality (AR) neuronavigation systems. The authors’ method, which makes use of the touchscreen capabilities of a tablet on which the AR navigation view is presented, enables surgeons to compensate for the effects of brainshift, misregistration, or tracking errors. They tested their system in a laboratory user study with ten subjects and found that they were able to achieve a median registration RMS error of 3.51 mm on landmarks around the craniotomy of interest. This is comparable to the level of accuracy attainable with previously proposed methods and currently available commercial systems while being simpler and quicker to use. The method could enable surgeons to quickly and easily compensate for most of the observed shift. Further advantages of their method include its ease of use, its small impact on the surgical workflow and its small-time requirement.
11 References
1.
Miner R.C.: ‘Image-guided neurosurgery’, J. Med. Imaging. Radiat. Sci., 2017, 48, (4), pp. 328–335 (10.1016/j.jmir.2017.06.005)
2.
Léger É., Drouin S., Louis Collins D., et al: ‘Quantifying attention shifts in augmented reality image-guided neurosurgery’, IET Healthc. Technol. Lett., 2017, 4, (5), pp. 188–192 (10.1049/htl.2017.0062)
3.
Cabrilo I., Bijlenga P., and Schaller K.: ‘Augmented reality in the surgery of cerebral aneurysms: a technical report’, Oper. Neurosurg., 2014, 10, (2), pp. 252–261 (10.1227/NEU.0000000000000328)
4.
Kersten-Oertel M., Gerard I.J., Drouin S., et al: ‘Augmented reality in neurovascular surgery: feasibility and first uses in the operating room’, Int. J. Comput. Assist. Radiol. Surg., 2015, 10, (11), pp. 1823–1836 (10.1007/s11548-015-1163-8)
5.
Tabrizi L.B., Mahvash M., Tabrizi L.B., et al: ‘Augmented reality: guided neurosurgery: accuracy and intraoperative application of an image projection technique’, J. Neurosurg., 2015, 123, (1), pp. 206–211 (10.3171/2014.9.JNS141001)
6.
Watanabe E., Satoh M., Konno T., et al: ‘The trans-visible navigator: a EEE-through neuronavigation system using augmented reality’, World Neurosurg., 2016, 87, pp. 399–405 (10.1016/j.wneu.2015.11.084)
7.
Stieglitz L.H., Fichtner J., Andres R., et al: ‘The silent loss of neuronavigation accuracy: a systematic retrospective analysis of factors influencing the mismatch of frameless stereotactic systems in cranial neurosurgery’, Neurosurgery, 2013, 72, (5), pp. 796–807 (10.1227/NEU.0b013e318287072d)
8.
Gerard I.J., Kersten-Oertel M., Petrecca K., et al: ‘Brain shift in neuronavigation of brain tumors: a review’, Med. Image Anal., 2017, 35, pp. 403–420 (10.1016/j.media.2016.08.007)
9.
Nabavi A., Black P.M., Gering D.T., et al: ‘Serial intraoperative MR imaging of brain shift’, Neurosurgery, 2001, 48, (4), pp. 787–798
10.
Rivaz H. and Louis Collins D.: Deformable registration of preoperative MR, pre-resection ultrasound, and post-resection ultrasound images of neurosurgery. Int. J. Comput. Assist. Radiol. Surg., 2015, 10, (7), pp. 1017–1028 (10.1007/s11548-014-1099-4)
11.
Nimsky C., Ganslandt O., Hastreiter P., et al: ‘Intraoperative compensation for brain shift’, Surg. Neurol., 2001, 56, (6), pp. 357–364 (10.1016/S0090-3019(01)00628-0)
12.
Pereira V.M., Smit-Ockeloen I., Brina O., et al: ‘Volumetric measurements of brain shift using intraoperative cone-beam computed tomography: preliminary study’, Operative Neurosurg., 12, (1), pp. 4–13 (10.1227/NEU.0000000000000999)
13.
Ruijters D., ter Haar Romeny B.M., and Suetens P.: ‘GPU-accelerated elastic 3D image registration for intra-surgical applications’, Comput. Methods Programs Biomed., 2011, 103, (2), pp. 104–112 (10.1016/j.cmpb.2010.08.014)
14.
Miga M.I., Sun K., Chen I., et al: ‘Clinical evaluation of a model-updated image-guidance approach to brain shift compensation: experience in 16 cases’, Int. J. Comput. Assist. Radiol. Surg., 2016, 11, (8), pp. 1467–1474 (10.1007/s11548-015-1295-x)
15.
Morin F., Courtecuisse H., Reinertsen I., et al: ‘Brain-shift compensation using intraoperative ultrasound and constraint-based biomechanical simulation’, Med. Image Anal., 2017, 40, pp. 133–153 (10.1016/j.media.2017.06.003)
16.
Drouin S., Kersten-Oertel M., and Louis Collins D.: ‘Interaction-based registration correction for improved augmented reality overlay in neurosurgery’. Workshop on Augmented Environments for Computer-Assisted Interventions, 2015, pp. 21–29
17.
Kantelhardt S.R., Gutenberg A., Neulen A., et al: ‘Video-assisted navigation for adjustment of image guidance accuracy to slight brain shift’, Operative Neurosurg., 2015, 11, (4), pp. 504–511 (10.1227/NEU.0000000000000921)
18.
Roethe A. L., Rösler J., Vajkoczy P., et al: ‘Current experiences and future perspectives for augmented reality visualization in navigated neurosurgical interventions’, 22nd Annual Conference of the International Society for Computer Aided Surgery, Berlin, Germany, June 2018
19.
Drouin S., Kochanowska A., Kersten-Oertel M., et al: ‘IBIS: an OR ready open-source platform for image-guided neurosurgery’, Int. J. Comput. Assist. Radiol. Surg., 2017, 12, (3), pp. 363–378 (10.1007/s11548-016-1478-0)
20.
Zhang Z.: ‘A flexible new technique for camera calibration’, IEEE Trans. Pattern Anal. Mach. Intell., 1999, 22, (11), pp. 1330–1334 (10.1109/34.888718)
21.
Tokuda J., Fischer G.S., Papademetris X., et al: ‘OpenIGTLink: an open network protocol for image-guided therapy environment’, Int. J. Med. Robot. Comput. Assist. Surg., 2009, 5, (4), pp. 423–434 (10.1002/rcs.274)
22.
Brooke J.: ‘Sus-a quick and dirty usability scale’, Usability Eval. Ind., 1996, 189, (194), pp. 4–7
Information & Authors
Information
Published in
Copyright
This is an open access article published by the IET under the Creative Commons Attribution-NonCommercial-NoDerivs License (http://creativecommons.org/licenses/by-nc-nd/3.0/)
History
Received: 10 August 2018
Accepted: 20 August 2018
Published online: 01 October 2018
Published in print: 19 October 2018
Inspec keywords
Keywords
- surgeon
- brainshift
- tracking errors
- median registration RMS error
- surgical workflow
- gesture-based registration correction
- mobile augmented reality image-guided neurosurgery system
- surgical tools
- GPS-like guidance
- image distortion
- patient-to-image alignment accuracy
- surgical procedure
- gesture-based method
- manual registration correction
- augmented reality neuronavigation systems
- preoperative images
- tablet touchscreen capability
- size 3.51 mm
Authors
Funding Information
Natural Sciences and Engineering Research Council of Canada: 0
Metrics & Citations
Metrics
Citations
Download article citation data for:
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download.
View Options
View options
Access content
Please select your options to get access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.