Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon openaccess Can surgical simulation be used to train detection and classification of neural networks?

Computer-assisted interventions (CAI) aim to increase the effectiveness, precision and repeatability of procedures to improve surgical outcomes. The presence and motion of surgical tools is a key information input for CAI surgical phase recognition algorithms. Vision-based tool detection and recognition approaches are an attractive solution and can be designed to take advantage of the powerful deep learning paradigm that is rapidly advancing image recognition and classification. The challenge for such algorithms is the availability and quality of labelled data used for training. In this Letter, surgical simulation is used to train tool detection and segmentation based on deep convolutional neural networks and generative adversarial networks. The authors experiment with two network architectures for image segmentation in tool classes commonly encountered during cataract surgery. A commercially-available simulator is used to create a simulated cataract dataset for training models prior to performing transfer learning on real surgical data. To the best of authors’ knowledge, this is the first attempt to train deep learning models for surgical instrument detection on simulated data while demonstrating promising results to generalise on real data. Results indicate that simulated data does have some potential for training advanced classification methods for CAI systems.

References

    1. 1)
      • 11. Lin, H.C., Shafran, I., Murphy, T.E., et al: ‘Automatic detection and segmentation of robot-assisted surgical motions’, Med. Image Comput. Comput. Assist. Interv., 2005, 8, (Pt 1), pp. 802810.
    2. 2)
    3. 3)
      • 22. Jia, Y., Shelhamer, E., Donahue, J., et al: ‘Caffe: convolutional architecture for fast feature embedding’. Proc. ACM Int. Conf. Multimedia (ACM), Orlando, November 2014, pp. 675678.
    4. 4)
      • 1. Maier-Hein, L., Vedula, S., Speidel, S., et al: ‘Surgical data science: enabling next-generation surgery’, arXiv:1701.06482, 2017.
    5. 5)
    6. 6)
    7. 7)
    8. 8)
    9. 9)
      • 16. Girshick, R., Donahue, J., Darrell, T., et al: ‘Rich feature hierarchies for accurate object detection and semantic segmentation’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR), Columbus, June 2014, pp. 580587.
    10. 10)
      • 18. Isola, P., Zhu, J.-Y., Zhou, T., et al: ‘Image-to-image translation with conditional adversarial networks’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR), Hawaii, July 2017.
    11. 11)
    12. 12)
      • 10. DiPietro, R., Lea, C., Malpani, A., et al: ‘Recognizing surgical activities with recurrent neural networks’, Medical Image Computing and Computer-Assisted Intervention, 2016 (LNCS, 9900), pp. 551558.
    13. 13)
      • 14. Long, J., Shelhamer, E., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Boston, October 2015, pp. 34313440.
    14. 14)
    15. 15)
    16. 16)
    17. 17)
      • 6. Katić, D., Wekerle, A.L., Gärtner, F., et al: ‘Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative context-aware assistance’. Information Processing in Computer-Assisted Interventions (IPCAI), 2014 (LNCS, 8498), pp. 158167.
    18. 18)
    19. 19)
    20. 20)
    21. 21)
      • 13. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘Imagenet classification with deep convolutional neural networks’. Advances in Neural Information Processing Systems (NIPS), 2012, pp. 10971105.
    22. 22)
      • 8. Rieke, N., Tan, D.J., Alsheakhali, M., et al: ‘Surgical tool tracking and pose estimation in retinal microsurgery’. Medical Image Computing and Computer-Assisted Intervention, 2015 (LNCS, 9349), pp. 266273.
    23. 23)
      • 20. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’. Int. Conf. Learning Representations (ICRL), San Diego, May 2015, pp. 114.
    24. 24)
      • 23. Eigen, D., Fergus, R.: ‘Predicting depth, surface normals and semantic labels with a common multi-scale convolutional architecture’. Proc. Int. Conf. Computer Vision (ICCV), Santiago, December 2015, pp. 26502658.
http://iet.metastore.ingenta.com/content/journals/10.1049/htl.2017.0064
Loading

Related content

content/journals/10.1049/htl.2017.0064
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address