Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon openaccess Evaluating salient object detection in natural images with multiple objects having multi-level saliency

This is an open access article published by the IET under the Creative Commons Attribution License (http://creativecommons.org/licenses/by-nc/3.0/)
Received 01/07/2019, Accepted 13/11/2019, Revised 21/09/2019, Published 15/11/2019

Salient object detection is evaluated using binary ground truth (GT) with the labels being salient object class and background. In this study, the authors corroborate based on three subjective experiments on a novel image dataset that objects in natural images are inherently perceived to have varying levels of importance. The authors' dataset, named SalMoN (saliency in multi-object natural images), has 588 images containing multiple objects. The subjective experiments performed record spontaneous attention and perception through eye fixation duration, point clicking and rectangle drawing. As object saliency in a multi-object image is inherently multi-level, they propose that salient object detection must be evaluated for the capability to detect all multi-level salient objects apart from the salient object class detection capability. For this purpose, they generate multi-level maps as GT corresponding to all the dataset images using the results of the subjective experiments, with the labels being multi-level salient objects and background. They then propose the use of mean absolute error, Kendall's rank correlation and average area under precision–recall curve to evaluate existing salient object detection methods on their multi-level saliency GT dataset. Approaches that represent saliency detection on images as local-global hierarchical processing of a graph perform well in their dataset.

Inspec keywords: image segmentation; object detection; statistical analysis; visual perception

Other keywords: object saliency; multilevel salient objects; image dataset; multiobject image; multiobject natural images; multilevel saliency GT dataset; dataset images; multilevel maps; salient object class detection capability

Subjects: Other topics in statistics; Other topics in statistics; Computer vision and image processing techniques; Optical, image and video signal processing

References

    1. 1)
      • 49. Puka, L.: ‘Kendall's tau’ in Lovric, M. (Ed.): International Encyclopedia of Statistical Science (Springer, Berlin Heidelberg, 2011), pp. 713–715.
    2. 2)
      • 22. Cheng, M., Warrell, J., Lin, W., et al: ‘Efficient salient region detection with soft image abstraction’. Proc. of IEEE Int. Conf. on Computer Vision (ICCV), Sydney, NSW, Australia, 2013.
    3. 3)
      • 16. Judd, T., Ehinger, K., Durand, F., et al: ‘Learning to predict where humans look’. Proc. of IEEE Int. Conf. on Computer Vision (ICCV), Kyoto, Japan, 2009, pp. 2106–2113.
    4. 4)
      • 13. Yarbus, A.L.: ‘Eye movements and vision’ (Springer, USA., 1967).
    5. 5)
      • 24. Shen, X., Wu, Y.: ‘A unified approach to salient object detection via low rank matrix recovery’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA., 2012, pp. 853–860.
    6. 6)
      • 45. Toet, A.: ‘Computational versus psychophysical bottom-up image saliency: a comparative evaluation study’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (11), pp. 2131–2146.
    7. 7)
      • 40. Chalmers, D.J., French, R.M., Hofstadter, D.R.: ‘High-level perception, representation, and analogy: a critique of artificial intelligence methodology’, J. Exp. Theor. Artif. Intell., 1992, 4, (3), pp. 185–211.
    8. 8)
      • 28. Zhang, J., Sclaroff, S., Lin, Z., et al: ‘Minimum barrier salient object detection at 80 fps’. IEEE Int. Conf. on Computer Vision, Las Condes, Chile, 2015, pp. 1404–1412.
    9. 9)
      • 7. Li, Y., Hou, X., Koch, C., et al: ‘The secrets of salient object segmentation’. IEEE Conf. on Computer Vision and Pattern Recognition, Columbus, OH, USA., 2014, pp. 280–287.
    10. 10)
      • 26. Jiang, B., Zhang, L., Lu, H., et al: ‘Saliency detection via absorbing markov chain’. Proc. of IEEE Int. Conf. on Computer Vision (ICCV), Sydney, NSW, Australia, 2013.
    11. 11)
      • 12. Henderson, J.M., Hollingworth, A.: ‘Eye movements during scene viewing: an overview’, in Eye guidance while reading and while watching dynamic scenes, (Elsevier, UK., 1998), pp. 269–293.
    12. 12)
      • 3. Han, J., Zhang, D., Cheng, G., et al: ‘Advanced deep-learning techniques for salient and category-specific object detection: a survey’, IEEE Signal Process. Mag., 2018, 35, (1), pp. 84–100.
    13. 13)
      • 19. Yildirim, G., Süsstrunk, S.: ‘FASA: Fast, accurate, and size-aware salient object detection’. Proc. of Asian Conf. on Computer Vision (ACCV), Singapore, 2014.
    14. 14)
      • 21. Jiang, H., Wang, J., Yuan, Z., et al: ‘Automatic salient object segmentation based on context and shape prior’. Proc. of BMVC, Dundee, UK., 2011, pp. 1–12.
    15. 15)
      • 32. Islam, A., Kalash, M., Bruce, N.D.B.: ‘Revisiting salient object detection: Simultaneous detection, ranking, and subitizing of multiple salient objects’. Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA., 2018.
    16. 16)
      • 18. Ruan, X., Tong, N., Lu, H.: ‘How far we away from a perfect visual saliency detection - DUT-OMRON: a new benchmark dataset’. Korea-Japan Joint Workshop on Frontiers of Computer Vision, Okinawa, Japan, 2014.
    17. 17)
      • 2. Niebur, E., Koch, C.: ‘Computational architectures for attention’, in Parasuraman, R. (ed.): ‘The attentive brain’ (MIT Press, Cambridge, MA, USA, 1998).
    18. 18)
      • 33. Liu, T., Yuan, Z., Sun, J., et al: ‘Learning to detect a salient object’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (2), pp. 353–367.
    19. 19)
      • 10. Zhou, Y., Mao, A., Huo, S., et al: ‘Salient object detection via fuzzy theory and object-level enhancement’, IEEE Trans. Multimed., 2019, 21, (1), pp. 74–85.
    20. 20)
      • 31. Huang, F., Qi, J., Lu, H., et al: ‘Salient object detection via multiple instance learning’, IEEE Trans. Image Process., 2017, 26, (4), pp. 1911–1922.
    21. 21)
      • 44. Engelke, U., Liu, H., Wang, J., et al: ‘Comparative study of fixation density maps’, IEEE Trans. Image Process., 2013, 22, (3), pp. 1121–1133.
    22. 22)
      • 47. Alexe, B., Deselaers, T., Ferrari, V.: ‘Measuring the objectness of image windows’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (11), pp. 2189–2202.
    23. 23)
      • 5. Alpert, S., Galun, M., Basri, R., et al: ‘Image segmentation by probabilistic bottom-up aggregation and cue integration’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Minneapolis, MN, USA., 2007, pp. 1–8.
    24. 24)
      • 36. Xiao, J., Ehinger, K.A., Hays, J., et al: ‘Sun database: exploring a large collection of scene categories’, Int. J. Comput. Vis., 2016, 119, (1), pp. 3–22.
    25. 25)
      • 14. Peters, R.J., Iyer, A., Itti, L., et al: ‘Components of bottom-up gaze allocation in natural images’, Vis. Res., 2005, 45, (18), pp. 2397–2416.
    26. 26)
      • 37. McMains, S., Kastner, S.: ‘Interactions of top-down and bottom-up mechanisms in human visual cortex’, J. Neurosci., 2011, 31, (2), pp. 587–597.
    27. 27)
      • 11. Chen, Y., Zou, W., Tang, Y., et al: ‘Scom: Spatiotemporal constrained optimization for salient object detection’, IEEE Trans. Image Process., 2018, 27, (7), pp. 3345–3357.
    28. 28)
      • 1. Baluch, F., Itti, L.: ‘Mechanisms of top-down attention’, Trends Neurosci., 2011, 34, (4), pp. 210–224.
    29. 29)
      • 6. Movahedi, V., Elder, J.H.: ‘Design and perceptual validation of performance measures for salient object segmentation’. Proc. of IEEE Computer Vision and Pattern Recognition (CVPR) Workshops, San Francisco, CA, USA., 2010, pp. 49–56.
    30. 30)
      • 23. Perazzi, F., Krahenbuhl, P., Pritch, Y., et al: ‘Saliency filters: Contrast based filtering for salient region detection’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA., 2012, pp. 733–740.
    31. 31)
      • 48. Alers, H., Bos, L., Heynderickx, I.: ‘How the task of evaluating image quality influences viewing behavior’. Proc. of Int. Workshop on Quality of Multimedia Experience, Mechelen, Belgium, 2011, pp. 167–172.
    32. 32)
      • 38. Deng, J., Dong, W., Socher, R., et al: ‘ImageNet: a large-scale hierarchical image database’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA., 2009.
    33. 33)
      • 27. Yildirim, G., Shaji, A., Süsstrunk, S.: ‘Saliency detection using regression trees on hierarchical image segments’. Proc. of IEEE Int. Conf. on Image Processing (ICIP), Paris, France, 2014.
    34. 34)
      • 8. Fan, D.P., Cheng, M.M., Liu, J.J., et al: ‘Salient objects in clutter: Bringing salient object detection to the foreground’. European Conf. on Computer Vision (ECCV), Munich, Germany, 2018,.
    35. 35)
      • 15. Wolfe, J.M.: ‘Guided search 2.0 a revised model of visual search’, Psychon. Bull. Rev., 1994, 1, (2), pp. 202–238.
    36. 36)
      • 9. Yang, C., Zhang, L., Lu, H., et al: ‘Saliency detection via graph-based manifold ranking’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA., 2013, pp. 3166–3173.
    37. 37)
      • 46. Ivory, M.Y., Hearst, M.A.: ‘Improving web site design’, IEEE Internet Comput., 2002, 6, (2), pp. 56–63.
    38. 38)
      • 39. Shenton, M.E., Turetsky, B.I.: ‘Understanding neuropsychiatric disorders: insights from neuroimaging’ (Cambridge University Press, UK., 2011).
    39. 39)
      • 4. Achanta, R., Hemami, S., Estrada, F., et al: ‘Frequency-tuned salient region detection’. Proc. of IEEE Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA., 2009, pp. 1597–1604.
    40. 40)
      • 20. Cheng, M., Zhang, G., Mitra, N.J., et al: ‘Global contrast based salient region detection’. Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA., 2011, pp. 409–416.
    41. 41)
      • 30. Peng, H., Li, B., Ling, H., et al: ‘Salient object detection via structured matrix decomposition’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 39, (4), pp. 818–832.
    42. 42)
      • 35. Everingham, M., Gool, L.V., Williams, C.K.I., et al: ‘The PASCAL Visual Object Classes Challenge 2010 (VOC2010) Results’. Available at http://www.pascal-network.org/challenges/VOC/voc2010/workshop/index.html.
    43. 43)
      • 29. Li, H., Lu, H., Lin, Z., et al: ‘Inner and inter label propagation: Salient object detection in the wild’, IEEE Trans. Image Process., 2015, 24, (10), pp. 3176–3186.
    44. 44)
      • 42. Itti, L., Koch, C., Niebur, E.: ‘A model of saliency-based visual attention for rapid scene analysis’, IEEE Trans. Pattern Anal. Mach. Intell., 1998, 20, (11), pp. 1254–1259.
    45. 45)
      • 41. Kandel, E.R., Schwartz, J.H., Jessell, T.M., et al: ‘Principles of neural science’ (McGraw-Hill, New York, USA, 2000, 4th edn.).
    46. 46)
      • 17. Li, J., Levine, M.D., An, X., et al: ‘Visual saliency based on scale-space analysis in the frequency domain’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (4), pp. 996–1010.
    47. 47)
      • 34. Martin, D., Fowlkes, C., Tal, D., et al: ‘A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics’. Proc. of IEEE Int. Conf. on Computer Vision (ICCV), Vancouver, BC, Canada, 2001, vol. 2, pp. 416–423.
    48. 48)
      • 43. Everingham, M., Eslami, S.M.A., Van Gool, L., et al: ‘The Pascal visual object classes challenge: a retrospective’, Int. J. Comput. Vis., 2014, 111, pp. 98–136.
    49. 49)
      • 25. Li, X., Li, Y., Shen, C., et al: ‘Contextual hypergraph modelling for salient object detection’. Proc. of IEEE Int. Conf. on Computer Vision (ICCV), Sydney, NSW, Australia, 2013.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2019.0787
Loading

Related content

content/journals/10.1049/iet-ipr.2019.0787
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
Print this page
This is a required field
Please enter a valid email address