Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Level set based shape prior and deep learning for image segmentation

© The Institution of Engineering and Technology
Received 11/12/2018, Accepted 17/10/2019, Revised 06/05/2019, Published 21/10/2019

Deep convolutional neural network can effectively extract hidden patterns in images and learn realistic image priors from the training set. And fully convolutional networks (FCNs) have achieved state-of-the-art performance in the image segmentation. However, these methods have the disadvantages of noise, boundary roughness and no prior shape. Therefore, this study proposes a level set with the deep prior method for the image segmentation based on the priors learned by FCNs. The FCNs can learn high-level semantic patterns from the training set. Also, the output of the FCNs represents the high-level semantic information as a probability map and the global affine transformation can obtain the optimal affine transformation of the intrinsic prior shape. Moreover, the improved level set method integrates the information of the original image, the probability map and the corrected prior shape to achieve the image segmentation. Compared with the traditional level set method of simple scenes, the proposed method solves the disadvantage of FCNs by using the high-level semantic information to segment images of complex scenes. Finally, Portrait data set are used to verify the effectiveness of the proposed method. The experimental results show that the proposed method can obtain more accurate segmentation results than the traditional FCNs.

Inspec keywords: affine transforms; learning (artificial intelligence); image segmentation; convolutional neural nets

Other keywords: fully convolutional networks; FCN; high-level semantic patterns; deep learning; portrait data set; image segmentation; intrinsic prior shape; specific image; improved level set method; deep prior method; convolutional neural network; training set; corrected prior shape; high-level semantic information; segmented images; realistic image priors

Subjects: Neural computing techniques; Knowledge engineering techniques; Optical, image and video signal processing; Computer vision and image processing techniques

References

    1. 1)
      • 14. Badrinarayanan, V., Kendall, A., Cipolla, R.: ‘SegNet: a deep convolutional encoder–decoder architecture for image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (12), pp. 2481–2495.
    2. 2)
      • 45. Chen, F., Yu, H., Hu, R., et al: ‘Deep learning shape priors for object segmentation’. 2013 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Portland, USA, 2013, pp. 1870–1877.
    3. 3)
      • 21. Liu, C., Chen, L.C., Schroff, F., et al: ‘Auto-DeepLab: hierarchical neural architecture search for semantic image segmentation’, arXiv preprint arXiv:190102985, 2019.
    4. 4)
      • 51. Lin, P., Zheng, C., Yang, Y., et al: ‘A probability model-based level set method for biomedical image segmentation’, J. X-Ray Sci. Technol., 2005, 13, (3), pp. 117–127.
    5. 5)
      • 49. Jia, Y., Shelhamer, E., Donahue, J., et al: ‘Caffe: convolutional architecture for fast feature embedding’, arXiv preprint arXiv:14085093, 2014.
    6. 6)
      • 41. Wakahara, T., Odaka, K.: ‘Adaptive normalization of handwritten characters using global/local affine transformation’, IEEE Trans. Pattern Anal. Mach. Intell., 1998, 20, (12), pp. 1332–1341.
    7. 7)
      • 38. Hubel, D.H., Wiesel, T.N.: ‘Receptive fields, binocular interaction and functional architecture in the cat's visual cortex’, J. Physiol., 1962, 160, (1), pp. 106–154.
    8. 8)
      • 30. Shen, X., Hertzmann, A., Jia, J., et al: ‘Automatic portrait segmentation for image stylization’. Computer Graphics Forum, 2016, vol. 35, pp. 93–102.
    9. 9)
      • 37. Kingma, D.P., Welling, M.: ‘Auto-encoding variational Bayes’, arXiv preprint arXiv:13126114, 2013.
    10. 10)
      • 1. Pal, N.R., Pal, S.K.: ‘A review on image segmentation techniques’, Pattern Recognit., 1993, 26, (9), pp. 1277–1294.
    11. 11)
      • 40. Liu, Z., Li, X., Luo, P., et al: ‘Semantic image segmentation via deep parsing network’. 2015 IEEE Int. Conf. on Computer Vision (ICCV), Santiago, Chile, 2015, pp. 1377–1385.
    12. 12)
      • 28. Hu, P., Shuai, B., Liu, J., et al: ‘Deep level sets for salient object detection’. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, 2017.
    13. 13)
      • 11. Munir, A., Soomro, S., Lee, C.H., et al: ‘Adaptive active contours based on variable kernel with constant initialisation’, IET Image Process., 2018, 12, pp. 1117–1123.
    14. 14)
      • 39. Zheng, S., Jayasumana, S., Romera-Paredes, B., et al: ‘Conditional random fields as recurrent neural networks’. Proc. of the IEEE Int. Conf. on Computer Vision, Santiago, Chile, 2015, pp. 1529–1537.
    15. 15)
      • 19. Chen, L.C., Zhu, Y., Papandreou, G., et al: ‘Encoder–decoder with atrous separable convolution for semantic image segmentation’, arXiv preprint arXiv:180202611, 2018.
    16. 16)
      • 34. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Las Vegas, USA, 2016, pp. 770–778.
    17. 17)
      • 8. Comaniciu, D., Meer, P.: ‘Mean shift: a robust approach toward feature space analysis’, IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24, (5), pp. 603–619.
    18. 18)
      • 2. LeCun, Y., Bengio, Y., Hinton, G.: ‘Deep learning’, Nature, 2015, 521, (7553), p. 436.
    19. 19)
      • 26. Wu, K., Yu, Y.: ‘Automatic object extraction from images using deep neural networks and the level-set method’, IET Image Process., 2018, 12, pp. 1131–1141.
    20. 20)
      • 6. Achanta, R., Shaji, A., Smith, K., et al: ‘SLIC superpixels compared to state-of-the-art superpixel methods’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (11), pp. 2274–2282.
    21. 21)
      • 24. Krähenbühl, P., Koltun, V.: ‘Efficient inference in fully connected CRFs with Gaussian edge potentials’. Advances in Neural Information Processing Systems, Granada, Spain, 2011, pp. 109–117.
    22. 22)
      • 3. Sezgin, M., Sankur, B.: ‘Survey over image thresholding techniques and quantitative performance evaluation’, J. Electron. Imaging, 2004, 13, (1), pp. 146–166.
    23. 23)
      • 5. Nguyen, H.T., Worring, M., Van Den Boomgaard, R.: ‘Watersnakes: energy-driven watershed segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2003, 25, (3), pp. 330–342.
    24. 24)
      • 17. Chen, Y., Li, J., Xiao, H., et al: ‘Dual path networks’. Advances in Neural Information Processing Systems, California, USA, 2017, pp. 4467–4475.
    25. 25)
      • 36. Zeiler, M.D., Taylor, G.W., Fergus, R.: ‘Adaptive deconvolutional networks for mid and high level feature learning’. 2011 IEEE Int. Conf. on Computer Vision (ICCV), Barcelona, Spain, 2011, pp. 2018–2025.
    26. 26)
      • 29. Tang, M., Valipour, S., Zhang, Z., et al: ‘A deep level set method for image segmentation’. Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Québec City, Canada, 2017, pp. 126–134.
    27. 27)
      • 12. Long, J., Shelhamer, E., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’. Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition, Boston, USA, 2015, pp. 3431–3440.
    28. 28)
      • 32. Harris, C., Stephens, M.: ‘A combined corner and edge detector’. Alvey Vision Conf., Manchester, USA, 1988, vol. 15, pp. 10–5244.
    29. 29)
      • 18. Wu, Z., Shen, C., Van Den Hengel, A.: ‘Wider or deeper: revisiting the ResNet model for visual recognition’, Pattern Recognit., 2019, 90, pp. 119–133.
    30. 30)
      • 20. Li, H., Xiong, P., Fan, H., et al: ‘DFANet: deep feature aggregation for real-time semantic segmentation’, arXiv preprint arXiv:190402216, 2019.
    31. 31)
      • 35. Zhou, B., Khosla, A., Lapedriza, A., et al: ‘Learning deep features for discriminative localization’. 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, 2016, pp. 2921–2929.
    32. 32)
      • 13. Ronneberger, O., Fischer, P., Brox, T.: ‘U-net: convolutional networks for biomedical image segmentation’. Int. Conf. on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015, pp. 234–241.
    33. 33)
      • 27. Ulyanov, D., Vedaldi, A., Lempitsky, V.: ‘Deep image prior’. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Utah, USA, 2018, pp. 9446–9454.
    34. 34)
      • 16. Geng, Q., Zhou, Z., Cao, X.: ‘Survey of recent progress in semantic image segmentation with CNNs’, Sci. China Inf. Sci., 2018, 61, (5), p. 051101.
    35. 35)
      • 23. Yuan, Y., Wang, J.: ‘OCNet: object context network for scene parsing’, arXiv preprint arXiv:180900916, 2018.
    36. 36)
      • 50. Everingham, M., Eslami, S.M.A., Van Gool, L., et al: ‘The PASCAL visual object classes challenge: a retrospective’, Int. J. Comput. Vis., 2015, 111, (1), pp. 98–136.
    37. 37)
      • 43. Malladi, R., Sethian, J.A., Vemuri, B.C.: ‘Shape modeling with front propagation: a level set approach’, IEEE Trans. Pattern Anal. Mach. Intell., 1995, 17, (2), pp. 158–175.
    38. 38)
      • 42. Hartley, R., Zisserman, A.: ‘Multiple view geometry in computer vision’ (Cambridge University Press, UK, 2003).
    39. 39)
      • 48. Li, C., Xu, C., Gui, C., et al: ‘Distance regularized level set evolution and its application to image segmentation’, IEEE Trans. Image Process., 2010, 19, (12), pp. 3243–3254.
    40. 40)
      • 4. Senthilkumaran, N., Rajesh, R.: ‘Edge detection techniques for image segmentation–a survey of soft computing approaches’, Int. J. Recent Trends Eng., 2009, 1, (2), pp. 250–254.
    41. 41)
      • 10. Chan, T.F., Vese, L.A.: ‘Active contours without edges’, IEEE Trans. Image Process., 2001, 10, (2), pp. 266–277.
    42. 42)
      • 31. Canny, J.: ‘A computational approach to edge detection’, in Martin, F., Oscar, F. (Eds.): ‘Readings in computer vision’ (Elsevier, USA, 1987), pp. 184–203.
    43. 43)
      • 44. Cremers, D., Schmidt, F.R., Barthel, F.: ‘Shape priors in variational image segmentation: convexity, Lipschitz continuity and globally optimal solutions’. 2008 IEEE Conf. on Computer Vision and Pattern Recognition. CVPR 2008, Anchorage, Alaska, 2008, pp. 1–6.
    44. 44)
      • 9. Kass, M., Witkin, A., Terzopoulos, D.: ‘Snakes: active contour models’, Int. J. Comput. Vis., 1988, 1, (4), pp. 321–331.
    45. 45)
      • 47. Li, C., Xu, C., Gui, C., et al: ‘Level set evolution without re-initialization: a new variational formulation’. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 2005. CVPR 2005, San Diego, USA, 2005, vol. 1, pp. 430–436.
    46. 46)
      • 7. Felzenszwalb, P.F., Huttenlocher, D.P.: ‘Efficient graph-based image segmentation’, Int. J. Comput. Vis., 2004, 59, (2), pp. 167–181.
    47. 47)
      • 46. Li, C., Kao, C.Y., Gore, J.C., et al: ‘Minimization of region-scalable fitting energy for image segmentation’, IEEE Trans. Image Process., 2008, 17, (10), pp. 1940–1949.
    48. 48)
      • 22. Fu, J., Liu, J., Tian, H., et al: ‘Dual attention network for scene segmentation’, arXiv preprint arXiv:180902983, 2018.
    49. 49)
      • 33. Zhou, Y., Ye, Q., Qiu, Q., et al: ‘Oriented response networks’. 2017 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Hawaii, USA, 2017, pp. 4961–4970.
    50. 50)
      • 15. Noh, H., Hong, S., Han, B.: ‘Learning deconvolution network for semantic segmentation’. Proc. IEEE Int. Conf. on Computer Vision, Santiago, Chile, 2015, pp. 1520–1528.
    51. 51)
      • 25. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs’, IEEE Trans. Pattern Anal. Mach. Intell., 2018, 40, (4), pp. 834–848.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.6622
Loading

Related content

content/journals/10.1049/iet-ipr.2018.6622
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address