access icon free Widening residual skipped network for semantic segmentation

© The Institution of Engineering and Technology
Received 18/01/2017, Accepted 14/05/2017, Revised 05/05/2017, Published 18/05/2017

Over the past two years deep convolutional neural networks have pushed the performance of computer vision systems to soaring heights on semantic segmentation. In this study, the authors present a novel semantic segmentation method of using a deep fully convolutional neural network to achieve image segmentation results with more precise boundary localisation. The above segmentation engine is trainable, and consists of an encoder network with widening residual skipped connections and a decoder network with a pixel-wise classification layer. Here the encoder network with widening residual skipped connections allows the combination of shallow layer features and deep layer semantic features, and the decoder network with classification layer maps the low-resolution encoder features to full resolution image with pixel-wise classification. The experimental results on PASCAL VOC 2012 semantic segmentation dataset and Cityscapes dataset show that the proposed method is effective and competitive.

Inspec keywords: image coding; image resolution; neural nets; image segmentation; computer vision; image classification

Other keywords: image segmentation; pixel-wise classification layer; deep convolutional neural networks; residual skipped connections; residual skipped network; PASCAL VOC 2012 semantic segmentation dataset; decoder network; Cityscapes dataset; low-resolution encoder; computer vision systems; classification layer maps; deep layer semantic features; precise boundary localisation; encoder network; resolution image

Subjects: Image and video coding; Neural computing techniques; Computer vision and image processing techniques

References

    1. 1)
      • 10. Zagoruyko, S., Komodakis, N.: ‘Wide residual networks’. British Machine Vision Conf., York, UK, September 2016, pp. 87.1–87.12.
    2. 2)
      • 7. Vemulapalli, R., Tuzel, O., Liu, M.Y., et al: ‘Gaussian conditional random field network for semantic segmentation’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, Nevada, June 2016, pp. 3224–3233.
    3. 3)
      • 25. Hariharan, B., Arbeláez, P., Girshick, R., et al: ‘Hypercolumns for object segmentation and finegrained localization’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, Massachusetts, June 2015, pp. 447–456.
    4. 4)
      • 1. Wu, C., Cheng, H.P., Li, S., et al: ‘Apesnet: a pixel-wise efficient segmentation network for embedded devices’, IET Cyber-Phys. Syst. Theory Appl., 2016, 1, (1), pp. 78–85.
    5. 5)
      • 22. Hariharan, B., Arbeláez, P., Bourdev, L., et al: ‘Semantic contours from inverse detectors’. Int. Conf. Computer Vision, Barcelona, Spain, November 2011, pp. 991–998.
    6. 6)
      • 20. Srivastava, N., Hinton, G.E., Krizhevsky, A., et al: ‘Dropout: a simple way to prevent neural networks from overfitting’, J. Mach. Learn. Res., 2014, 15, (1), pp. 1929–1958.
    7. 7)
      • 18. Zilly, J.G., Srivastava, R.K., Koutník, J., et al: ‘Recurrent highway networks’, arXiv preprint arXiv:1607.03474, 2016.
    8. 8)
      • 19. Huang, G., Sun, Y., Liu, Z., et al: ‘Deep networks with stochastic depth’. In European Conf. on Computer Vision, Amsterdam, The Netherlands, October 2016, pp. 646–661.
    9. 9)
      • 17. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, (99), pp. 1–1.
    10. 10)
      • 12. Noh, H., Hong, S., Han, B.: ‘Learning deconvolution network for semantic segmentation’. Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, December 2015, pp. 1520–1528.
    11. 11)
      • 27. Lin, G., Shen, C., van den Hengel, A., et al: ‘Efficient piecewise training of deep structured models for semantic segmentation’. Proc. of IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, Nevada, June 2016, pp. 3194–3203.
    12. 12)
      • 8. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for largescale image recognition’. arXiv preprint arXiv:1409.1556, 2014.
    13. 13)
      • 26. Mostajabi, M., Yadollahpour, P., Shakhnarovich, G.: ‘Feedforward semantic segmentation with zoom-out features’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, Massachusetts, June 2015, pp. 3376–3385.
    14. 14)
      • 21. Everingham, M., Van Gool, L., Williams, C.K., et al: ‘The pascal visual object classes (VOC) challenge’, Int. J. Comput. Vis., 2010, 88, (2), pp. 303–338.
    15. 15)
      • 14. Mahendran, A., Vedaldi, A.: ‘Understanding deep image representations by inverting them’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, Massachusetts, June 2015, pp. 5188–5196.
    16. 16)
      • 23. Cordts, M., Omran, M., Ramos, S., et al: ‘The cityscapes dataset for semantic urban scene understanding’. Proc. of IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, Nevada, June 2016, pp. 3213–3223.
    17. 17)
      • 28. Uhrig, J., Cordts, M., Franke, U., et al: ‘Pixel-level encoding and depth layering for instance-level semantic labeling’. German Conf. on Pattern Recognition, Hannover, Germany, September 2016, pp. 14–25.
    18. 18)
      • 15. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘Imagenet classification with deep convolutional neural networks’. Advances in Neural Information Processing Systems, Lake Tahoe, Nevada, December 2012, pp. 1097–1105.
    19. 19)
      • 2. Girshick, R., Donahue, J., Darrell, T., et al: ‘Rich feature hierarchies for accurate object detection and semantic segmentation’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Columbus, Ohio, 2014, pp. 580–587.
    20. 20)
      • 11. Badrinarayanan, V., Kendall, A., Cipolla, R.: ‘Segnet: a deep convolutional encoder-decoder architecture for image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, (99), pp. 1–1.
    21. 21)
      • 24. Vedaldi, A., Lenc, K.: ‘Matconvnet: convolutional neural networks for MATLAB’. Proc. 23rd ACM Int. Conf. Multimedia, Queensland, Australia, October 2015, pp. 689–692.
    22. 22)
      • 9. He, K., Zhang, X., Ren, S., et al: ‘Deep residual learning for image recognition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Las Vegas, Nevada, June 2016, pp. 770–778.
    23. 23)
      • 13. Zeiler, M.D.: ‘Visualizing and understanding convolutional networks’. Proc. European Conf. Computer Vision, September 2014, pp. 818–833.
    24. 24)
      • 5. Liu, Z., Li, X., Luo, P., et al: ‘Semantic image segmentation via deep parsing network’. Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, December 2015, pp. 1377–1385.
    25. 25)
      • 16. Zheng, S., Jayasumana, S., Romera-Paredes, B., et al: ‘Conditional random fields as recurrent neural networks’. Proc. IEEE Int. Conf. Computer Vision, Santiago, Chile, December 2015, pp. 1529–1537.
    26. 26)
      • 3. Fu, R., Li, B., Gao, Y., et al: ‘Fully automatic figure-ground segmentation algorithm based on deep convolutional neural network and GrabCut’, IET Image Process., 2016, 10, (12), pp. 937–942.
    27. 27)
      • 4. Long, J., Shelhamer, E., Darrell, T.: ‘Fully convolutional networks for semantic segmentation’. Proc. IEEE Conf. Computer Vision and Pattern Recognition, Boston, Massachusetts, June 2015, pp. 3431–3440.
    28. 28)
      • 6. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Semantic image segmentation with deep convolutional nets and fully connected crfs’, Comput. Sci., 2014, (4), pp. 357–361.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2017.0070
Loading

Related content

content/journals/10.1049/iet-ipr.2017.0070
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading