access icon free Deep convolutional neural networks for automatic segmentation of left ventricle cavity from cardiac magnetic resonance images

This work conducts a feasibility study of deep learning approaches for automatic segmentation of left ventricle (LV) cavity from cardiac magnetic resonance (CMR) images. Automatic LV cavity segmentation is a challenging task, partially due to the small size of the object as compared to the large CMR image background, especially at the apex. To cater for small object segmentation, the authors present a localisation-segmentation framework, to first locate the object in the large full image, then segment the object within the small cropped region of interest. The localisation is performed by a deep regression model based on convolutional neural networks, while the segmentation is done by the deep neural networks based on U-Net architecture. They also employ the Dice loss function for the training process of the segmentation models, to investigate its effects on the segmentation performance. The deep learning models are trained and evaluated by using public endocardium-annotated CMR datasets from York University and MICCAI 2009 LV Challenge websites. The average dice metric values of the authors’ proposed framework are 0.91 and 0.93, respectively, on these two databases. These results are promising as compared to the best results achieved by the current state-of-art, which shows the potentials of deep learning approaches for this particular application.

Inspec keywords: regression analysis; learning (artificial intelligence); image segmentation; neural nets; medical image processing; object detection; biomedical MRI; convolution; cardiovascular system

Other keywords: deep convolutional neural networks; deep neural network based U-Net architecture; localisation-segmentation framework; automatic left ventricle cavity segmentation framework; object segmentation framework; public endocardium-annotated CMR datasets; regression model based convolutional neural networks; cardiac magnetic resonance images; Dice loss function; deep learning approaches

Subjects: Medical magnetic resonance imaging and spectroscopy; Knowledge engineering techniques; Biology and medical computing; Biomedical magnetic resonance imaging and spectroscopy; Optical, image and video signal processing; Patient diagnostic methods and instrumentation; Computer vision and image processing techniques

References

    1. 1)
      • 27. Long, J., Shelhamer, E., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’, 2015. Available at http://arxiv.org/abs/1411.4038.
    2. 2)
      • 35. Li, Y., Qi, H., Dai, J., et al: ‘Fully convolutional instance-aware semantic segmentation’, 2016. Available at https://arxiv.org/abs/1611.07709.
    3. 3)
      • 22. van Tulder, G., de Bruijne, M.: ‘Learning features for tissue classification with the classification restricted Boltzmann machine’. MCV, 2014 (LNCS, 8848), pp. 4758.
    4. 4)
      • 18. Chen, T., Chefdhotel, C.: ‘Deep learning based automatic immune cell detection for immunohistochemistry images’. 5th Int. Workshop on Machine Learning in Medical Imaging (MLMI'14), Boston, MA, USA, 2014, pp. 1724.
    5. 5)
      • 26. Ngo, T.A., Carneiro, G.: ‘Left ventricle segmentation from cardiac MRI combining level set methods with deep belief networks’. IEEE Int. Conf. Image Processing (ICIP), Melbourne, Australia, 2013, pp. 695699.
    6. 6)
      • 30. Yang, X.L., Like, G., Yeo, S.Y., et al: ‘Automatic segmentation of left ventricular myocardium by deep convolutional and de-convolutional neural networks’. Computing in Cardiology (CINC 2016), 2016, pp. 8184.
    7. 7)
      • 20. Dhungel, N., Carneiro, G., Bradley, A.P.: ‘Deep structured learning for mass segmentation from mammograms’, 2014. Available at http://arxiv.org/abs/1410.7454.
    8. 8)
      • 2. Kang, D., Woo, J., Slomka, P.J., et al: ‘Heart chambers and whole heart segmentation techniques: review’, SPIE J. Electron. Imag., 2012, 21, (1), pp. 131139.
    9. 9)
      • 23. Greenspan, H., van Ginneken, B., Summers, R.M.: ‘Guest editorial – deep learning in medical imaging: overview and future promise of an exciting new techniques’, IEEE Trans. Med. Imag., 2016, 35, (5), pp. 11531159.
    10. 10)
      • 14. Cruz-Roa, A., Basavanhally, A., Gonzalez, F., et al: ‘Automatic detection of invasive ductal carcinoma in whole slide images with convolutional neural networks’. SPIE Medical Imaging, vol. 9041, 2014, doi: 10.1117/12.2043872.
    11. 11)
      • 21. Yang, X.L., Yeo, S.Y., Hong, J.M., et al: ‘A deep learning approach for tumor tissue image classification’. IASTED Biomedical Engineering, proceeding, 832, 2016, DOI: 10.2316/P.2016.832-025.
    12. 12)
      • 28. Tran, P.V.: ‘A fully convolutional neural network for cardiac segmentation in short-axis MRI’, 2016. Available at https://arxiv.org/abs/1604.00494.
    13. 13)
      • 19. Dhungel, N., Carneiro, G., Bradley, A.P.: ‘Deep learning and structured prediction for the segmentation of mass in mammograms’. Medical Image Computing and Computer Assisted Intervention (MICCAI 2015), 2015 (LNCS, 9349), pp. 605612.
    14. 14)
      • 17. Esteva, A., Kuprel, B., Thrun, S.: ‘Deep networks for early stage skin disease and skin cancer classification’. Project Report, Stanford University, 2015.
    15. 15)
      • 12. Ciresan, D.C., Giusti, A., Gambardella, L., et al: ‘Mitosis detection in breast cancer histology images with deep neural networks’. Medical Image Computing and Computer Assisted Interventions (MICCAI 2013), 2013 (LNCS, 8150), pp. 411418.
    16. 16)
      • 5. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, pp. 436444.
    17. 17)
      • 41. Zheng, S., Jayasumana, S., Romera-Paredes, B., et al: ‘Conditional random fields as recurrent neural networks’, 2015. Available at https://arxiv.org/abs/1502.03240.
    18. 18)
      • 24. Avendi, M.R., Kheradvar, A., Jafarkhani, H.: ‘A combined deep-learning and deformable-model approach to fully automatic segmentation of the left ventricle in Cardiac MRI, 2015. Available at http://arxiv.org/abs/1512.07951.
    19. 19)
      • 15. Cruz-Roa, A., Arevalo, J., Madabhushi, A., et al: ‘A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection’. Medical Image Computing and Computer-Assisted Intervention (MICCAI 2013), 2013 (LNCS, 8150), pp. 403410.
    20. 20)
      • 39. Noh, H., Hong, S., Han, B.: ‘Learning deconvolution network for semantic segmentation’, 2015. Available at http://arxiv.org/abs/1505.04366.
    21. 21)
      • 16. Ciresan, D.C., Giusti, A., Gambardella, L.M., et al: ‘Deep neural networks segment neuronal membranes in electron microscopy images’. Neural Information Processing Systems (NIPS 2012), 2012, pp. 28432851.
    22. 22)
      • 47. Korlev, S., Safiullin, A., Belyaev, M., et al: ‘Residual and plain convolutional neural networks for 3D brain MRI classification’, 2017. Available at https://arxiv.org/abs/1701.06643.
    23. 23)
      • 4. Bengio, Y., Courville, A., Vincent, P.: ‘Representation learning: a review and new perspectives’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (8), pp. 17982828.
    24. 24)
      • 33. Andreopoulos, A., Tsotsos, J.K.: ‘Efficient and generalizable statistical models of shape and appearance for analysis of cardiac MRI’, Med. Image Anal., 2008, 12, (3), pp. 335357. Available at http://www.cse.yorku.ca/~mridataset/.
    25. 25)
      • 8. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘ImageNet classification with deep convolutional neural networks’. Neural Information Processing Systems (NIPS 2012), 2012, pp. 11061114.
    26. 26)
      • 32. Drozdzal, M., Vorontsov, E., Chartrand, G., et al: ‘The importance of skip connections in biomedical image segmentation’, 2016. Available at https://arxiv.org/abs/1608.04117.
    27. 27)
      • 43. U-Net Kera Implementation, 2016. Available at https://github.com/jocicmarko/ultrasound-nerve-segmentation.
    28. 28)
      • 1. Petitjean, C., Dacher, J.N.: ‘A review of segmentation methods in short axis cardiac MR images’, Med. Image Anal., 2011, 15, (2), pp. 169184.
    29. 29)
      • 25. Emad, O., Yassine, I.A., Fahmy, A.S.: ‘Automatic localization of the left ventricle in cardiac MRI images using deep learning’. IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy, 2015, pp. 683686.
    30. 30)
      • 11. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’, 2015. Available at http://arxiv.org/abs/1512.03385.
    31. 31)
      • 13. Malon, C., Cosatto, E.: ‘Classification of mitotic figures with convolutional neural networks and seeded blob features’, J. Pathol. Inf., 2013, 4, (1), p. 9.
    32. 32)
      • 7. LeCun, Y., Bottou, L., Bengio, Y., et al: ‘Gradient-based learning applied to document recognition’, Proc. IEEE, 1998, 86, (11), pp. 22782324.
    33. 33)
      • 40. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Semantic image segmentation with deep convolutional nets and fully connected CRFs’, 2014. Available at https://arxiv.org/abs/1412.7062.
    34. 34)
      • 10. Szegedy, C., Liu, W., Jia, Y., et al: ‘Going deeper with convolutions’, 2014. Available at https://arxiv.org/abs/1409.4842.
    35. 35)
      • 37. Petitjean, C., Zuluaga, M.A., Bai, W., et al: ‘Right ventricle segmentation from cardiac MRI: a collation study’, Med. Image Anal., 2015, 19, (1), pp. 287202.
    36. 36)
      • 46. Kleesiek, J., Urban, G., Hubert, A., et al: ‘Deep MRI brain extraction: a 3D convolutional neural networks for skull stripping’, NeuroImage, 2016, 129, pp. 460469.
    37. 37)
      • 45. Kingma, D., Ba, J.: ‘Adam: a method for stochastic optimization’, 2014. Available at http://arxiv.org/abs/1412.6980v8.
    38. 38)
      • 9. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, 2014. Available at http://arxiv.org/abs/1409.1556.
    39. 39)
      • 36. He, K., Gkioxari, G., Dollar, P., et al: ‘Mask R-CNN’, 2017. Available at https://arxiv.org/abs/1703.06870.
    40. 40)
      • 3. Cocosco, C., Niessen, W., Netsch, T., et al: ‘Automatic image- driven segmentation of the ventricles in cardiac cine MRI’, J. Magn. Reson. Imag., 2008, 28, (2), pp. 366374.
    41. 41)
      • 6. Schmidhuber, J.: ‘Deep learning in neural networks: an overview’, Neural Netw., 2015, 61, pp. 85117.
    42. 42)
      • 29. Ronneberger, O., Fischer, P., Brox, T.: ‘U-Net: convolutional networks for biomedical image segmentation’, 2015. Available at https://arxiv.org/abs/1505.04597.
    43. 43)
      • 38. Yang, X.L., Su, Y., Duan, R., et al: ‘Cardiac image segmentation by random walks with dynamic information’, IET Comput. Vis., 2016, 10, (1), pp. 7986.
    44. 44)
      • 34. Radau, P., Lu, Y., Connelly, K., et al: ‘Evaluation framework for algorithms segmenting short axis cardiac MRI’. MICCAI 2009 Workshop on Cardiac MR Left Ventricle Segmentation Challenge, 2009. Available at http://smial.sri.utoronto.ca/LV_Challenge/Home.html.
    45. 45)
      • 42. Yu, F., Koltun, V.: ‘Multi-scale context aggregation by dilated convolutions’, 2015. Available at https://arxiv.org/abs/1511.07122.
    46. 46)
      • 31. Milletari, F., Navab, N., Ahmadi, S.A.: ‘V-Net: fully convolutional neural networks for volumetric medical image segmentation’, 2016. Available at https://arxiv.org/abs/1606.04797.
    47. 47)
      • 44. Deep Learning for Cardiac Segmentation, 2017. Available at https://bitbucket.org/ihpc_cs/med_seg.git.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2016.0482
Loading

Related content

content/journals/10.1049/iet-cvi.2016.0482
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading