access icon free Iterative PET image reconstruction using cascaded data consistency generative adversarial network

This study proposed a GAN-based reconstruction method-cascaded data consistency generative adversarial network (CDCGAN) to recover high-quality PET images from filtered back projection PET images with streaking artifacts and high noise. First, the authors embed defined data consistency layer (DC layer) in their generator network to constrain the reconstruction process and adjust accurately generated faked PET images. Second, to improve the accuracy of reconstruction on average, their generator network was built iteratively to achieve better performance with simple structures. They observed that the proposed CDCGAN allows the preservation of fine anomalous features while eliminating the streaking artifacts and noise. Experimental results show that the reconstructed PET images by their methods perform well comparably to other state-of-the-art methods but at a faster speed. A clinical experiment was also performed to show the validity of the CDCGAN for artifacts reduction.

Inspec keywords: image reconstruction; positron emission tomography; medical image processing; iterative methods

Other keywords: GAN-based reconstruction method-cascaded data consistency generative adversarial network; reconstructed PET images; high-quality PET images; reconstruction process; iterative PET image reconstruction; streaking artifacts; generator network; filtered back projection PET images

Subjects: Interpolation and function approximation (numerical analysis); Optical, image and video signal processing; Biology and medical computing; Computer vision and image processing techniques; Numerical approximation and analysis; Nuclear medicine, emission tomography; Nuclear medicine, emission tomography; Interpolation and function approximation (numerical analysis); Patient diagnostic methods and instrumentation

References

    1. 1)
      • 53. Johnson, J., Alahi, A., Fei-Fei, L.: ‘Perceptual losses for real-time style transfer and super-resolution’. European Conf. on Computer Vision, Cham, 2016, pp. 694711.
    2. 2)
      • 47. Denton, E.L., Chintala, S., Fergus, R.: ‘Deep generative image models using a laplacian pyramid of adversarial networks’. Advances in Neural Information Processing Systems, Montreal, Quebec, Canada, 2015, pp. 14861494.
    3. 3)
      • 25. Gong, K., Guan, J., Liu, C.C., et al: ‘Pet image denoising using a deep neural network through fine tuning’, IEEE Transactions on Radiation and Plasma Medical Sciences, 2018, 3, (2), pp. 153161.
    4. 4)
      • 70. Wang, G.: ‘A perspective on deep imaging’, IEEE Access, 2016, 4, pp. 89148924.
    5. 5)
      • 11. Kim, H., Chen, J., Wang, A., et al: ‘Non-local total-variation (NLTV) minimization combined with reweighted L1-norm for compressed sensing CT reconstruction’, Phys. Med. Biol., 2016, 61, (18), p. 6878.
    6. 6)
      • 20. Würfl, T., Ghesu, F.C., Christlein, V., et al: ‘Deep learning computed tomography’. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Cham, 2016, pp. 432440.
    7. 7)
      • 15. Valiollahzadeh, S., Clark, J., Mawlawi, O.: ‘MO-G-17A-05: PET image deblurring using adaptive dictionary learning’, Med. Phys., 2014, 41, (6Part25), pp. 437438.
    8. 8)
      • 49. Wang, X., Gupta, A.: ‘Generative image modeling using style and structure adversarial networks’. European Conf. on Computer Vision, Cham, 2016, pp. 318335.
    9. 9)
      • 69. Wang, G., Kalra, M., Orton, C.G.: ‘Machine learning will transform radiology significantly within the next 5 years’, Med. Phys., 2017, 44, (6), pp. 20412044.
    10. 10)
      • 63. Kim, K., Dutta, J., Groll, A., et al: ‘Penalized maximum likelihood reconstruction of ultrahigh resolution PET with depth of interaction’. The 13th Int. Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine, Philadelphia, PA, USA, 2015, pp. 296299.
    11. 11)
      • 66. Ronneberger, O., Fischer, P., Brox, T.: ‘U-net: convolutional networks for biomedical image segmentation’. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Cham, 2015, pp. 234241.
    12. 12)
      • 37. Nie, D., Trullo, R., Lian, J., et al: ‘Medical image synthesis with context-aware generative adversarial networks’. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Cham, 2017, pp. 417425.
    13. 13)
      • 12. Yang, X.: ‘Interventional molecular imaging’, Radiology, 2010, 254, (3), pp. 651654.
    14. 14)
      • 33. Zhu, J.Y., Krähenbühl, P., Shechtman, E., et al: ‘Generative visual manipulation on the natural image manifold’. European Conf. on Computer Vision, Cham, 2016, pp. 597613.
    15. 15)
      • 61. Dong, H., Supratak, A., Mai, L., et al: ‘Tensorlayer: a versatile library for efficient deep learning development’. Proc. of the 25th ACM Int. Conf. on Multimedia, New York, NY, USA, 2017, pp. 12011204.
    16. 16)
      • 44. Xie, Z., Baikejiang, R., Li, T., et al: ‘Generative adversarial network based regularized image reconstruction for PET’, The Fifteenth International Meeting on Fully Three-Dimensional Image Reconstruction in Radiology and Nuclear Medicine. Tours, France, 2019.
    17. 17)
      • 35. Zhao, J., Zhang, J., Li, Z.., et al: ‘DD-cycleGAN: unpaired image dehazing via double-discriminator cycle-consistent generative adversarial network’, Eng. Appl. Artif. Intell., 2019, 82, pp. 263271.
    18. 18)
      • 62. Gu, K., Li, L., Lu, H., et al: ‘A fast reliable image quality predictor by fusing micro-and macro-structures’, IEEE Trans. Ind. Electron., 2017, 64, (5), pp. 39033912.
    19. 19)
      • 46. Gauthier, J.: ‘Conditional generative adversarial nets for convolutional face generation’, Class Project for Stanford CS231N: Convolutional Neural Networks for Visual Recognition, Winter semester, 2014, 2014, (5), p. 2.
    20. 20)
      • 40. Yang, G., Yu, S., Dong, H., et al: ‘DAGAN: deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction’, IEEE Trans. Med. Imaging, 2017, 37, (6), pp. 13101321.
    21. 21)
      • 17. Gu, K., Xia, Z., Qiao, J., et al: ‘Deep dual-channel neural network for image-based smoke detection’, IEEE Trans. Multimed., 2019, 22, (2), pp. 311323.
    22. 22)
      • 54. Yang, Q., Yan, P., Kalra, M.K., et al: ‘CT image denoising with perceptive deep neural networks’, arXiv preprint arXiv:1702.07019, 2017.
    23. 23)
      • 56. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, arXiv preprint arXiv:1409.1556, 2014.
    24. 24)
      • 13. Buades, A., Coll, B., Morel, J.M.: ‘A non-local algorithm for image denoising’. 2005 IEEE Computer Society Conf. on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, USA, 2005, vol. 2, pp. 6065.
    25. 25)
      • 29. Ledig, C., Theis, L., Huszár, F., et al: ‘Photo-realistic single image super-resolution using a generative adversarial network’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 46814690.
    26. 26)
      • 58. Russakovsky, O., Deng, J., Su, H., et al: ‘Imagenet large scale visual recognition challenge’, Int. J. Comput. Vis., 2015, 115, (3), pp. 211252.
    27. 27)
      • 5. Balda, M., Hornegger, J., Heismann, B.: ‘Ray contribution masks for structure adaptive sinogram filtering’, IEEE Trans. Med. Imaging, 2012, 31, (6), pp. 12281239.
    28. 28)
      • 42. Humm, J.L., Rosenfeld, A., Del Guerra, A.: ‘From PET detectors to PET scanners’, European journal of nuclear medicine and molecular imaging, 2003, 30, (11), pp. 15741597.
    29. 29)
      • 71. Prakash, P., Kalra, M.K., Kambadakone, A.K., et al: ‘Reducing abdominal CT radiation dose with adaptive statistical iterative reconstruction technique’, Investigative radiology, 2010, 45, (4), pp. 202210.
    30. 30)
      • 45. Schlemper, J., Caballero, J., Hajnal, J.V., et al: ‘A deep cascade of convolutional neural networks for dynamic MR image reconstruction’, IEEE Trans. Med. Imaging, 2017, 37, (2), pp. 491503.
    31. 31)
      • 32. Brock, A., Lim, T., Ritchie, J.M., et al: ‘Neural photo editing with introspective adversarial networks’, arXiv preprint arXiv:1609.07093, 2016.
    32. 32)
      • 26. Wu, D., Kim, K., Fakhri, G.E., et al: ‘A cascaded convolutional neural network for x-ray low-dose CT image denoising’, arXiv preprint arXiv:1705.04267, 2017.
    33. 33)
      • 8. Lange, K., Carson, R.: ‘EM reconstruction algorithms for emission and transmission tomography’, J Comput Assist Tomogr, 1984, 8, (2), pp. 30616.
    34. 34)
      • 4. Tao, X., Zhang, H., Wang, Y., et al: ‘VVBP-tensor in the FBP algorithm: its properties and application in low-dose CT reconstruction’, IEEE Trans. Med. Imaging, 2020, 39, (3), pp. 764776.
    35. 35)
      • 24. Xu, J., Gong, E., Pauly, J., et al: ‘200x low-dose PET reconstruction using deep learning’, arXiv preprint arXiv:1712.04119, 2017.
    36. 36)
      • 55. Ledig, C., Theis, L., Huszár, F., et al: ‘Photo-realistic single image super-resolution using a generative adversarial network’, arXiv preprint arXiv:1609.04802, 2016.
    37. 37)
      • 19. Dong, C., Loy, C.C., He, K., et al: ‘Image super-resolution using deep convolutional networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 38, (2), pp. 295307.
    38. 38)
      • 27. Goodfellow, I., Pouget-Abadie, J., Mirza, M., et al: ‘Generative adversarial nets’. Advances in Neural Information Processing Systems, British Columbia, Canada, 2014, pp. 26722680.
    39. 39)
      • 39. Du, Q., Yang, W., Wang, Y., et al: ‘DRGAN: a deep residual generative adversarial network for PET image reconstruction’, IET Image Process., 2020, 14, (9).
    40. 40)
      • 6. Manduca, A., Yu, L., Trzasko, J.D., et al: ‘Projection space denoising with bilateral filtering and CT noise modeling for dose reduction in CT’, Med. Phys., 2009, 36, (11), pp. 49114919.
    41. 41)
      • 59. He, K., Zhang, X., Ren, S., et al: ‘Delving deep into rectifiers: surpassing human-level performance on imagenet classification’. Proc. of the IEEE Int. Conf. on Computer Vision, NW Washington, DC, USA, 2015, pp. 10261034.
    42. 42)
      • 14. Dabov, K., Foi, A., Katkovnik, V., et al: ‘Image denoising with block-matching and 3D filtering’. Image Processing: Algorithms and Systems, Neural Networks, and Machine Learning. Int. Society for Optics and Photonics, Quebec City, Canada, 2006, 6064, 606414.
    43. 43)
      • 43. Spuhler, K., Serrano-Sosa, M., Cattell, R., et al: ‘Full-count PET Recovery from Low-count Image Using a Dilated Convolutional Neural Network’, arXiv preprint arXiv:1910.11865, 2019.
    44. 44)
      • 65. Mathieu, M., Couprie, C., LeCun, Y.: ‘Deep multi-scale video prediction beyond mean square error’, arXiv preprint arXiv:1511.05440, 2015.
    45. 45)
      • 68. Dutta, J., Leahy, R.M., Li, Q.: ‘Non-local means denoising of dynamic PET images’, PloS one, 2013, 8, (12), p. e81390.
    46. 46)
      • 41. Hornik, K., Stinchcombe, M., White, H.: ‘Multilayer feedforward networks are universal approximators’, Neural Netw., 1989, 2, (5), pp. 359366.
    47. 47)
      • 57. Huang, X., Li, Y., Poursaeed, O., et al: ‘Stacked generative adversarial networks’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 50775086.
    48. 48)
      • 38. Wolterink, J.M., Leiner, T., Viergever, M.A., et al: ‘Generative adversarial networks for noise reduction in low-dose CT’, IEEE Trans. Med. Imaging, 2017, 36, (12), pp. 25362545.
    49. 49)
      • 2. Hunt, B.R.: ‘Image Reconstruction from Projections: Implementation and Applications’ (GT Herman, Editor), 1981.
    50. 50)
      • 7. Wang, J., Li, T., Lu, H., et al: ‘Penalized weighted least-squares approach to sinogram noise reduction and image reconstruction for low-dose X-ray computed tomography’, IEEE Trans. Med. Imaging, 2006, 25, (10), pp. 12721283.
    51. 51)
      • 23. Xiang, L., Qiao, Y., Nie, D., et al: ‘Deep auto-context convolutional neural networks for standard-dose PET image estimation from low-dose PET/MRI’, Neurocomputing, 2017, 267, pp. 406416.
    52. 52)
      • 50. Glorot, X., Bordes, A., Bengio, Y.: ‘Deep sparse rectifier neural networks’. Proc. of the Fourteenth Int. Conf. on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA, 2011, pp. 315323.
    53. 53)
      • 48. Reed, S., Akata, Z., Yan, X., et al: ‘Generative adversarial text to image synthesis’, arXiv preprint arXiv:1605.05396, 2016.
    54. 54)
      • 16. Le Pogam, A., Hanzouli, H., Hatt, M., et al: ‘Denoising of PET images by combining wavelets and curvelets for improved preservation of resolution and quantitation’, Med. Image Anal., 2013, 17, (8), pp. 877891.
    55. 55)
      • 64. Socher, R., Ganjoo, M., Manning, C.D., et al: ‘Zero-shot learning through cross-modal transfer’. Advances in Neural Information Processing Systems, Lake Tahoe, NV, USA, 2013, pp. 935943.
    56. 56)
      • 10. Sauer, K., Bouman, C.: ‘A local update strategy for iterative reconstruction from projections’, IEEE Trans. Signal Process., 1993, 41, (2), pp. 534548.
    57. 57)
      • 3. Tang, X., Ning, R.: ‘A cone beam filtered backprojection (CB-FBP) reconstruction algorithm for a circle-plus-two-arc orbit’, Med. Phys., 2001, 28, (6), p. 1042.
    58. 58)
      • 18. Gu, K., Zhang, Y., Qiao, J.: ‘Ensemble meta learning for few-shot soot density recognition’, IEEE Trans. Ind. Inf., 2020, (99), p. 1.
    59. 59)
      • 51. He, K., Zhang, X., Ren, S., et al: ‘Identity mappings in deep residual networks’. European Conf. on Computer Vision, Cham, 2016, pp. 630645.
    60. 60)
      • 36. Wang, J., Li, J., Sun, B., et al: ‘SAR image synthesis based on conditional generative adversarial networks’, The Journal of Engineering, 2019, 2019, (21), pp. 80938097.
    61. 61)
      • 34. Isola, P., Zhu, J.Y., Zhou, T., et al: ‘Image-to-image translation with conditional adversarial networks’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Honolulu, HI, USA, 2017, pp. 11251134.
    62. 62)
      • 21. Chen, H., Zhang, Y., Kalra, M.K., et al: ‘Low-dose CT with a residual encoder-decoder convolutional neural network’, IEEE Trans. Med. Imaging, 2017, 36, (12), pp. 25242535.
    63. 63)
      • 60. Kingma, D.P., Adam, B.J.: ‘A method for stochastic optimisation’, arXiv preprint arXiv:1412.6980, 2014.
    64. 64)
      • 22. Jiao, J., Ourselin, S.: ‘Fast PET reconstruction using Multi-scale Fully Convolutional Neural Networks’, arXiv preprint arXiv:1704.07244, 2017.
    65. 65)
      • 1. Alessio, A., Kinahan, P.: ‘PET image reconstruction’, Nuclear Medicine, 2006, 1, pp. 122.
    66. 66)
      • 30. Qiao, J., Song, H., Zhang, K., et al: ‘Image super-resolution using conditional generative adversarial network’, IET Image Process., 2019, 13, pp. 26732679.
    67. 67)
      • 28. Mirza, M., Osindero, S.: ‘Conditional generative adversarial nets’, arXiv preprint arXiv:1411.1784, 2014.
    68. 68)
      • 67. Yu, S., Muhammed, H.H.: ‘Noise type evaluation in positron emission tomography images’. 2016 1st Int. Conf. on Biomedical Engineering (IBIOMED), Yogyakarta, Indonesia, 2016, pp. 16.
    69. 69)
      • 52. Shi, W., Caballero, J., Huszár, F., et al: ‘Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, NV, USA, 2016, pp. 18741883.
    70. 70)
      • 31. Zareapoor, M., Celebi, M.E., Yang, J.: ‘Diverse adversarial network for image super-resolution’, Signal Process., Image Commun., 2019, 74, pp. 191200.
    71. 71)
      • 9. Li, T., Li, X., Wang, J., et al: ‘Nonlinear sinogram smoothing for low-dose X-ray CT’, IEEE Trans. Nucl. Sci., 2004, 51, (5), pp. 25052513.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2020.1056
Loading

Related content

content/journals/10.1049/iet-ipr.2020.1056
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading