access icon free Bag-of-features for image memorability evaluation

Image memorability represents the degree to which images are remembered or forgotten after a period of time. Studying image memorability in computer vision is the task of finding special characteristics in memorable images, in order to develop a representative model of this type of images. Several approaches have been realised to examine features that can affect image memorability. In this study, the authors use bag-of-features as another kind of visual feature descriptor to assess image memorability. The authors’ method based on bag-of-visual-words (BoVWs) technique involves four main steps. First, the authors extract local image features from regions/points of interest which are automatically detected. Then, they encode these local features by mapping them to a created visual vocabulary. Later, the authors apply features pooling and normalisation techniques to obtain image BoVW representation. Finally, the authors use this representation to examine image memorability as a problem of classification. They present different implementation choices for each step and compare reached results. The authors’ method performs best significant results in comparison with other approaches found in literature.

Inspec keywords: computer vision; feature extraction; image classification

Other keywords: image memorability evaluation; bag-of-visual-words; pooling technique; BoVW technique; normalisation technique; visual vocabulary; bag-of-features; computer vision; local image feature extraction; representative model; visual feature descriptor; classification problem

Subjects: Computer vision and image processing techniques; Image recognition

References

    1. 1)
      • 7. Jégou, H., Matthijs, D., Cordelia, S.: ‘Improving bag-of-features for large scale image search’, Int. J. Comput. Vis., 2010, 87, (3), pp. 316336.
    2. 2)
      • 31. Burges, C.: ‘A tutorial on support vector machines for pattern recognition’, Data Min. Knowl. Discov., 1998, pp. 121167.
    3. 3)
      • 13. Bylinskii, Z., Isola, P., Bainbridge, C., et al: ‘Intrinsic and extrinsic effects on image memorability’, Vis. Res., 2015, 116, pp. 165178.
    4. 4)
      • 32. Hsu, C.-W., Lin, C.-J.: ‘A comparison of methods for multiclass support vector machines’, IEEE Trans. Neural Netw., 2002, 13, (2), pp. 415425.
    5. 5)
      • 2. Isola, P., Xiao, J., Torralba, A., et al: ‘What makes an image memorable?’. CVPR, Colorado Springs, CO, USA, June 2011, pp. 145152.
    6. 6)
      • 24. Lloyd, S.: ‘Least square quantization in PCM’, IEEE Trans. Inf. Theory, 1982, 28, (2), pp. 129137.
    7. 7)
      • 10. Mancas, M., Le Meur, O.: ‘Memorability of natural scene: the role of attention’. Proc. IEEE Int. Conf. on Image Process., Melbourne, Australia, September 2013, pp. 196200.
    8. 8)
      • 16. Harris, C., Stephens, M.: ‘A combined corner and edge detector’. Proc. Fourth Alvey Vision Conf., Manchester, UK, September 1988, pp. 147151.
    9. 9)
      • 5. Lazebnik, S., Cordelia, S., Jean, P.: ‘Beyond bags of features: spatial pyramid matching for recognizing natural scene categories’. CVPR, New York, NY, USA, June 2006, pp. 21692178.
    10. 10)
      • 17. Tuytelaars, T., van Gool, L.: ‘Matching widely separated views based on affine invariant regions’, Int. J. Comput. Vis., 2004, 59, (1), pp. 6185.
    11. 11)
      • 3. Isola, P., Parikh, D., Torralba, A., et al: ‘Understanding the intrinsic memorability of images’. NIPS, Granada, Spain, December 2011, pp. 24292437.
    12. 12)
      • 11. Celikkale, B., Erdem, A., Erdem, E.: ‘Visual attention-driven spatial pooling for image memorability’. IEEE Computer Vision and Pattern Recognition Workshops, Portland, OR, USA, June 2013, pp. 976983.
    13. 13)
      • 33. Breiman, L., Friedman, J.H., Olshen, R.A., et al: ‘Classification and regression trees’. (Wadsworth & Brooks, Cole Advanced Books & Software, Monterey, CA, 1984).
    14. 14)
      • 19. Jurie, F., Triggs, B.: ‘Creating efficient codebooks for visual recognition’. Proc. Tenth IEEE Int. Conf. Computer Vision, Beijing, China, October 2005, pp. 604610.
    15. 15)
      • 26. Lee, H., Battle, A., Raina, R., et al: ‘Efficient sparse coding algorithms’. NIPS, Vancouver, British Columbia, Canada, December 2006, pp. 801808.
    16. 16)
      • 28. Lingxi, X., Qi, T., Bo, Z.: ‘Feature normalization for part-based image classification’. ICIP, Melbourne, Australia, September 2013, pp. 26072611.
    17. 17)
      • 30. Altman, N.S.: ‘An introduction to kernel and nearest-neighbor nonparametric regression’, Am. Stat., 1992, 46, (3), pp. 175185.
    18. 18)
      • 25. Csurka, G., Bray, C., Dance, C., et al: ‘Visual categorization with bags of keypoints’. ECCV Int. Workshop on Statistical Learning in Computer Vision, Prague, 2004, pp. 122.
    19. 19)
      • 20. Lowe, D.G.: ‘Distinctive image features from scale-invariant key points’, Int. J. Comput. Vis., 2004, 60, (2), pp. 91110.
    20. 20)
      • 1. Rensink, R.A., O'Regan, J.K., Clark, J.J.: ‘To see or not to see: the need for attention to perceive changes in scenes’, Psych. Sci., 1997, 8, (5), pp. 368373.
    21. 21)
      • 6. Nowak, E., Frédéric, J., Bill, T.: ‘Sampling strategies for bag-of-features image classification’. ECCV, Graz, Austria, 2006, pp. 490503.
    22. 22)
      • 12. Kim, J., Yoon, S., Pavlovic, V.: ‘Relative spatial features for image memorability’. Twenty- First ACM Int. Conf. Multimedia Proc., Barcelona, Spain, October 2013, pp. 761764.
    23. 23)
      • 15. Mikolajczyk, K., Schmid, C.: ‘Scale & affine invariant interest point detectors’, Int. J. Comput. Vis., 2004, 60, (1), pp. 6386.
    24. 24)
      • 21. Ke, Y., Sukthankar, R.: ‘PCA-SIFT: a more distinctive representation for local image descriptors’. CVPR, 2004, vol. 2, pp. 506513.
    25. 25)
      • 9. Isola, P., Xiao, J., Parikh, D., et al: ‘What makes a photograph memorable?’, IEEE Trans. Pattern Anal., 2014, 36, (7), pp. 14691482.
    26. 26)
      • 34. Xiao, J., Hayes, J., Ehinger, K., et al: ‘Sun database: large-scale scene recognition from abbey to zoo’. CVPR, San Francisco, CA, USA, June 2010, pp. 34853492.
    27. 27)
      • 23. Oliva, A., Torralba, A.: ‘Modeling the shape of the scene: a holistic representation of the spatial envelope’, Int. J. Comput. Vis., 2001, 42, (3), pp. 145175.
    28. 28)
      • 22. Bay, H., Tuytelaars, T., Gool, L.V.: ‘Surf: speeded up robust features’. Proc. Ninth ECCV, Graz, Austria, May 2006, pp. 404417.
    29. 29)
      • 14. Wang, J.J., Bensmail, H., Xin, G.: ‘Joint learning and weighting of visual vocabulary for bag-of-feature based tissue classification’, Pattern Recognit., 2013, 46, (12), pp. 32493255.
    30. 30)
      • 27. Wang, J., Yang, J., Yu, K., et al: ‘Locality-constrained linear coding for image classification’. CVPR, San Francisco, CA, USA, June 2010, pp. 33603367.
    31. 31)
      • 8. Khosla, A., Xiao, J., Torralba, A., et al: ‘Memorability of image regions’. NIPS, Lake Tahoe, Nevada, United States, December 2012, vol. 25, pp. 305313.
    32. 32)
      • 35. Chang, C.C., Lin, C.J.: ‘LIBSVM: a library for support vector machines’, ACM Trans. Intell. Syst. Technol., 2011, 2, (3), pp. 127.
    33. 33)
      • 4. Tuytelaars, T., Mikolajczyk, K.: ‘Local invariant feature detectors: a survey’, Found. Trends Comput. Graph. Vis., 2007, 3, (3), pp. 177280.
    34. 34)
      • 18. Fogel, I., Sagi, D.: ‘Gabor filters as texture discriminator’, Biol. Cybern., 1989, 61, (2), pp. 103113.
    35. 35)
      • 29. Ng, A.Y., Jordan, M.I.: ‘On discriminative vs. generative classifiers: a comparison of logistic regression and I Bayes’. NIPS, Vancouver, British Columbia, Canada, December 2001, pp. 841848.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2015.0383
Loading

Related content

content/journals/10.1049/iet-cvi.2015.0383
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading