http://iet.metastore.ingenta.com
1887

Automatic object extraction from images using deep neural networks and the level-set method

Automatic object extraction from images using deep neural networks and the level-set method

For access to this article, please select a purchase option:

Buy article PDF
$19.95
(plus tax if applicable)
Buy Knowledge Pack
10 articles for $120.00
(plus taxes if applicable)

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership 

Recommend Title Publication to library

You must fill out fields marked with: *

Librarian details
Name:*
Email:*
Your details
Name:*
Email:*
Department:*
Why are you recommending this title?
Select reason:
 
 
 
 
 
IET Image Processing — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

The authors propose an automatic method for extracting objects with fine quality from photographs. The authors’ method starts with finding bounding boxes that enclose potential objects, which is achievable by state-of-the-art object proposal methods. To further segment objects within obtained bounding boxes, the authors propose a new multi-pass level-set method based on saliency detection and foreground pixel classification. The level-set function is initially constructed with respect to the automatically detected salient parts within the bounding box, which eliminates potential user interaction and predicts an initial set of pixels on the object. The input features for foreground pixel classifiers are constructed as a combination of classical texture features from the Gabor filter banks and convolutional features from a pre-trained deep neural network. Through multi-pass evolution of the level-set function and re-training of the foreground pixel classifier, the authors’ method is able to overcome possible inaccuracies in the initial level-set function and converge to the real object boundary.

References

    1. 1)
      • 1. Chen, T., Cheng, M.M., Tan, P., et al: ‘Sketch2photo: internet image montage’, ACM Trans. Graph., 2009, 28, (5), pp. 124:1124:10.
    2. 2)
      • 2. Lalonde, J.F., Hoiem, D., Efros, A.A., et al: ‘Photo clip art’, ACM Trans. Graph., 2007, 26, (3), p. 3.
    3. 3)
      • 3. Kuo, W., Hariharan, B., Malik, J.: ‘Deepbox: learning objectness with convolutional networks’. Proc. 2015 IEEE Int. Conf. on Computer Vision, Washington, DC, USA, 2015, pp. 24792487.
    4. 4)
      • 4. Ghodrati, A., Diba, A., Pedersoli, M., et al: ‘Deepproposal: hunting objects by cascading deep convolutional layers’. Proc. 2015 IEEE Int. Conf. on Computer Vision, 2015, pp. 25782586.
    5. 5)
      • 5. Ren, S., He, K., Girshick, R., et al: ‘Faster R-CNN: towards real-time object detection with region proposal networks’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (6), pp. 11371149.
    6. 6)
      • 6. Maninis, K.K., Pont-Tuset, J., Arbelaez, P., et al: ‘Convolutional oriented boundaries: from image segmentation to high-level tasks’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, PP, (99), p. 1.
    7. 7)
      • 7. Lempitsky, V., Kohli, P., Rother, C., et al: ‘Image segmentation with a bounding box prior’. Proc. 2009 IEEE Int. Conf. on Computer Vision, 2009, pp. 277284.
    8. 8)
      • 8. Rother, C., Kolmogorov, V., Blake, A.: ‘’Grabcut’: interactive foreground extraction using iterated graph cuts’, ACM Trans. Graph., 2004, 23, (3), pp. 309314.
    9. 9)
      • 9. Arbelaez, P., Maire, M., Fowlkes, C., et al: ‘Contour detection and hierarchical image segmentation’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (5), pp. 898916.
    10. 10)
      • 10. Boykov, Y.Y., Jolly, M.P.: ‘Interactive graph cuts for optimal boundary amp: region segmentation of objects in N-D images’. Proc. 2001 IEEE Int. Conf. on Computer Vision, 2001, pp. 105112.
    11. 11)
      • 11. Mortensen, E.N., Barrett, W.A.: ‘Intelligent scissors for image composition’. Proc. 22nd Annual Conf. on Computer Graphics and Interactive Techniques, New York, NY, USA, 1995, pp. 191198.
    12. 12)
      • 12. Li, Y., Sun, J., Tang, C.K., et al: ‘Lazy snapping’, ACM Trans. Graph., 2004, 23, (3), pp. 303308.
    13. 13)
      • 13. Protiere, A., Sapiro, G.: ‘Interactive image segmentation via adaptive weighted distances’, IEEE Trans. Image Process., 2007, 16, (4), pp. 10461057.
    14. 14)
      • 14. Liu, J., Sun, J., Shum, H.Y.: ‘Paint selection’, ACM Trans. Graph., 2009, 28, (3), pp. 69:169:7.
    15. 15)
      • 15. Pont-Tuset, J., Arbelaez, P., Barron, J.T., et al: ‘Multiscale combinatorial grouping for image segmentation and object proposal generation’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (1), pp. 128140.
    16. 16)
      • 16. Yu, Y., Fang, C., Liao, Z.: ‘Piecewise flat embedding for image segmentation’. Proc. 2015 IEEE Int. Conf. on Computer Vision, 2015, pp. 13681376.
    17. 17)
      • 17. Farabet, C., Couprie, C., Najman, L., et al: ‘Learning hierarchical features for scene labeling’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (8), pp. 19151929.
    18. 18)
      • 18. Noh, H., Hong, S., Han, B.: ‘Learning deconvolution network for semantic segmentation’. Proc. 2015 IEEE Int. Conf. on Computer Vision, 2015, pp. 15201528.
    19. 19)
      • 19. Dai, J., He, K., Li, Y., et al: ‘Instance-sensitive fully convolutional networks’. Proc. of the 2016 European Conf. on Computer Vision, 2016, pp. 534549.
    20. 20)
      • 20. Chen, L.C., Papandreou, G., Kokkinos, I., et al: ‘Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, PP, (99), p. 1.
    21. 21)
      • 21. He, K., Gkioxari, G., Dollar, P., et al: ‘Mask R-CNN’. Proc. 2017 IEEE Int. Conf. on Computer Vision, 2017, pp. 29802988.
    22. 22)
      • 22. Erhan, D., Szegedy, C., Toshev, A., et al: ‘Scalable object detection using deep neural networks’. Proc. 2014 IEEE Conf. on Computer Vision and Pattern Recognition, 2014, pp. 21552162.
    23. 23)
      • 23. Szegedy, C., Toshev, A., Erhan, D.: ‘Deep neural networks for object detection’, Adv. Neural Inf. Process. Syst., 2013, 26, pp. 25532561.
    24. 24)
      • 24. Girshick, R.: ‘Fast R-CNN’. Proc. 2015 IEEE Int. Conf. on Computer Vision, 2015, pp. 14401448.
    25. 25)
      • 25. Pinheiro, P.O., Collobert, R., Dollar, P.: ‘Learning to segment object candidates’, Adv. Neural Inf. Process. Syst., 2015, 28, pp. 19901998.
    26. 26)
      • 26. Pinheiro, P.O., Lin, T.Y., Collobert, R., et al: ‘Learning to refine object segments’. Proc. 2016 European Conf. on Computer Vision, Cham, 2016, pp. 7591.
    27. 27)
      • 27. Milletari, F., Navab, N., Ahmadi, S.A.: ‘V-Net: fully convolutional neural networks for volumetric medical image segmentation’. Proc. 2016 Int. Conf. on 3D Vision, 2016, pp. 565571.
    28. 28)
      • 28. Osher, S., Sethian, J.A.: ‘Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations’, J. Comput. Phys., 1988, 79, (1), pp. 1249.
    29. 29)
      • 29. Osher, S., Fedkiw, R.: ‘The level set methods and dynamic implicit surfaces’, Appl. Mech. Rev., 2004, 57, p. 273.
    30. 30)
      • 30. Adalsteinsson, D., Sethian, J.A.: ‘A fast level set method for propagating interfaces’, J. Comput. Phys., 1995, 118, (2), pp. 269277.
    31. 31)
      • 31. Li, C., Xu, C., Gui, C., et al: ‘Distance regularized level set evolution and its application to image segmentation’, IEEE Trans. Image Process., 2010, 19, (12), pp. 32433254.
    32. 32)
      • 32. Paragios, N., Deriche, R.: ‘Geodesic active regions for supervised texture segmentation’. Proc. 1999 IEEE Int. Conf. on Computer Vision, 1999, pp. 926932.
    33. 33)
      • 33. Chan, T.F., Vese, L.A.: ‘Active contours without edges’, IEEE Trans. Image Process., 2001, 10, (2), pp. 266277.
    34. 34)
      • 34. Leventon, M.E., Grimson, W.E.L., Faugeras, O.: ‘Statistical shape influence in geodesic active contours’. Proc. 2000 IEEE Conf. on Computer Vision and Pattern Recognition, 2000, pp. 316323.
    35. 35)
      • 35. Leventon, M.E., Faugeras, O., Grimson, W.E.L., et al: ‘Level set based segmentation with intensity and curvature priors’. Proc. 2000 IEEE Workshop on Mathematical Methods in Biomedical Image Analysis, 2000, pp. 411.
    36. 36)
      • 36. Chen, Y., Tagare, H.D., Thiruvenkadam, S., et al: ‘Using prior shapes in geometric active contours in a variational framework’, Int. J. Comput. Vis., 2002, 50, (3), pp. 315328.
    37. 37)
      • 37. Rousson, M., Paragios, N.: ‘Shape priors for level set representations’. Proc. 2002 European Conf. on Computer Vision, Heidelberg, Berlin, 2002, pp. 7892.
    38. 38)
      • 38. Bresson, X., Vandergheynst, P., Thiran, J.P.: ‘A variational model for object segmentation using boundary information and shape prior driven by the Mumford-Shah functional’, Int. J. Comput. Vis., 2006, 68, (2), pp. 145162.
    39. 39)
      • 39. Liu, Y., Yu, Y.: ‘Interactive image segmentation based on level sets of probabilities’, IEEE Trans. Vis. Comput. Graphics, 2012, 18, (2), pp. 202213.
    40. 40)
      • 40. Li, G., Yu, Y.: ‘Deep contrast learning for salient object detection’. Proc. 2016 IEEE Conf. on Computer Vision and Pattern Recognition, 2016, pp. 478487.
    41. 41)
      • 41. Yang, J., Price, B., Cohen, S., et al: ‘Object contour detection with a fully convolutional encoder–decoder network’. Proc. 2016 IEEE Conf. on Computer Vision and Pattern Recognition, 2016, pp. 193202.
    42. 42)
      • 42. Canny, J.: ‘A computational approach to edge detection’, IEEE Trans. Pattern Anal. Mach. Intell., 1986, 8, (6), pp. 679698.
    43. 43)
      • 43. Xu, C., Prince, J.L.: ‘Snakes, shapes, and gradient vector flow’, IEEE Trans. Image Process., 1998, 7, (3), pp. 359369.
    44. 44)
      • 44. Uijlings, J.R.R., van de Sande, K.E.A., Gevers, T., et al: ‘Selective search for object recognition’, Int. J. Comput. Vis., 2013, 104, (2), pp. 154171.
    45. 45)
      • 45. Zitnick, C.L., Dollár, P.: ‘Edge boxes: locating object proposals from edges’. Proc. 2014 European Conf. on Computer Vision, Cham, 2014, pp. 391405.
    46. 46)
      • 46. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’, CoRR, 2014, abs/1409.1556.
    47. 47)
      • 47. Olshausen, B.A., Field, D.J.: ‘Sparse coding with an overcomplete basis set: a strategy employed by V1?’, Vis. Res., 1997, 37, (23), pp. 33113325.
    48. 48)
      • 48. Moller, M.F.: ‘A scaled conjugate gradient algorithm for fast supervised learning’, Neural Netw., 1993, 6, (4), pp. 525533.
    49. 49)
      • 49. Martin, D., Fowlkes, C., Tal, D., et al: ‘A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics’. Proc. 2001 IEEE Int. Conf. on Computer Vision, 2001, pp. 416423.
    50. 50)
      • 50. Everingham, M., Van Gool, L., Williams, C.K.I., et al: ‘The Pascal visual object classes (VOC) challenge’, Int. J. Comput. Vis., 2010, 88, (2), pp. 303338.
    51. 51)
      • 51. Lin, T.Y., Maire, M., Belongie, S., et al: ‘Microsoft COCO: common objects in context’. Proc. 2014 European Conf. on Computer Vision, Cham, 2014, pp. 740755.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2017.1144
Loading

Related content

content/journals/10.1049/iet-ipr.2017.1144
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address