access icon free Boosting landmark retrieval baseline with burstiness detection

© The Institution of Engineering and Technology
Received 26/12/2016, Accepted 12/11/2017, Revised 10/10/2017, Published 16/11/2017

In image retrieval, the bag-of-visual-words model-based approaches combined withthe spatial verification (SP) post-processing step have achieved considerableprogress. However, in practice, especially for retrieving landmark images, theauthors have observed that this baseline suffers from the problem of burst matches.This issue is caused by repetitive visual patterns that appear frequently amongimages. Local features derived from these burst patterns can redundantly matchothers, resulting in many invalid matches that vote over-estimated similarity scoresfor irrelevant images. Essentially, this problem can be mainly attributed to tworeasons, (i) the non-exclusive matching leads to one-to-many matches, (ii) the SPfails to filter burst matches that are closely located. To tackle this problem, aburstiness detection approach using geometric and visual word information of localfeatures is proposed. Firstly, a geometric filtering strategy is employed to removematches that are not consistent with global scale variation. Then, the one-to-onematching strategy is applied to detect and eliminate one-to-many matches. Finally,a down-weighting burstiness strategy is adopted to penalise the voting weight ofburst matches. Experimental results on three public datasets demonstrate that theproposed approach can achieve a comparable or even better accuracy over otherpopular approaches.

Inspec keywords: filtering theory; image matching; image retrieval

Other keywords: global scale variation; landmark image retrieval; non-exclusive matching; local features; voting weight; SP; bag-of-visual-words model-based approaches; spatial verification post-processing step; burstiness detection approach; down-weighting burstiness strategy; irrelevant images; boosting landmark retrieval baseline; burst matches; burst patterns; geometric filtering strategy

Subjects: Information retrieval techniques; Computer vision and image processing techniques

References

    1. 1)
      • 2. Schönberger, J.L., Radenović, F., Chum, O., et al: ‘From single image query to detailed 3d reconstruction’. 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston, USA, June 2015, pp. 5126–5134.
    2. 2)
      • 33. Philbin, J., Chum, O., Isard, M., et al: ‘Lost in quantization: improving particular object retrieval in large scale image databases’. 2008 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Alaska, USA, June 2008, pp. 1–8.
    3. 3)
      • 8. Krizhevsky, A., Sutskever, I., Hinton, G.E.: ‘Imagenet classification with deep convolutional neural networks’. inPereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (Eds.): ‘Advances in neural information processing systems 25’ (Curran Associates, Inc., New York, USA, 2012), pp. 1097–1105.
    4. 4)
      • 24. Eva, M., Kevin, M., Noel, E.O., et al: ‘Bags of local convolutional features for scalable instance search’. Proc. 2016 ACM Int. Conf. on Multimedia Retrieval, New York, NY, USA, 2016, pp. 327–331. Available at http://doi.acm.org/10.1145/2911996.2912061.
    5. 5)
      • 14. Shen, X., Lin, Z., Brandt, J., et al: ‘Spatially-constrained similarity measure for large-scale object retrieval’, IEEE Trans. Pattern Anal. Mach. Intell., 2014, 36, (6), pp. 1229–1241.
    6. 6)
      • 31. Yuan, Y., Ma, D., Wang, Q.: ‘Hyperspectral anomaly detection by graph pixel selection’, IEEE Trans. Cybern., 2016, 46, (12), pp. 3123–3134.
    7. 7)
      • 19. Chum, O., Mikulík, A., Perdoch, M., et al: ‘Total recall II: query expansion revisited’. 2011 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, USA, June 2011, pp. 889–896.
    8. 8)
      • 6. Sivic, J., Zisserman, A.: ‘Video google: efficient visual search of videos’ (Springer, Berlin, Heidelberg, 2006), pp. 127–144. Available at http://dx.doi.org/10.1007/11957959_7.
    9. 9)
      • 29. Yuan, Y., Wang, Q., Zhu, G.: ‘Fast hyperspectral anomaly detection via high-order 2-d crossing filter’, IEEE Trans. Geosci. Remote Sens., 2015, 53, (2), pp. 620–630.
    10. 10)
      • 12. Radenović, F., Tolias, G., Chum, O.: ‘CNN image retrieval learns from BoW: unsupervised fine-tuning with hard examples’. 14th European Conf. on Computer Vision, Amsterdam, The Netherlands, October 2016, pp. 3–20.
    11. 11)
      • 18. Chum, O., Philbin, J., Sivic, J., et al: ‘Total recall: automatic query expansion with a generative feature model for object retrieval’. 2007 IEEE 11th Int. Conf. on Computer Vision, Rio de Janeiro, Brazil, October 2007, pp. 1–8.
    12. 12)
      • 4. Arandjelovic, R., Gronat, P., Torii, A., et al: ‘Netvlad: CNN architecture for weakly supervised place recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, PP, (99), pp. 1–1.
    13. 13)
      • 22. Babenko, A., Slesarev, A., Chigorin, A., et al: in Computer Vision - ECCV 2014: 13th European ConferenceFleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (Eds.): ‘Neural codes for image retrieval’ (Springer Int. Publishing, Cham, 2014), pp. 584–599.
    14. 14)
      • 9. Yandex, A.B., Lempitsky, V.: ‘Aggregating local deep features for image retrieval’. 2015 IEEE Int. Conf. on Computer Vision (ICCV), Santiago, Chile, December 2015, pp. 1269–1277.
    15. 15)
      • 10. Tolias, G., Sicre, R., Jégou, H.: ‘Particular object retrieval with integral max-pooling of CNN activations’, arXiv preprint arXiv:151105879, 2015.
    16. 16)
      • 20. Lowe, D.G.: ‘Distinctive image features from scale-invariant keypoints’, Int. J. Comput. Vis., 2004, 60, (2), pp. 91–110. Available at https://doi.org/10.1023/B:VISI.0000029664.99615.94.
    17. 17)
      • 25. Chum, O., Matas, J.I., Obdrzalek, S.: ‘Enhancing RANSAC by generalized model optimization’. Proc. Asian Conf. on Computer Vision, Jeju, Korea, January 2004.
    18. 18)
      • 13. Jégou, H., Douze, M., Schmid, C.: ‘On the burstiness of visual elements’. 2009 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Florida, USA, June 2009, pp. 1169–1176.
    19. 19)
      • 27. Schönberger, J.L., Price, T., Sattler, T., et al: Computer Vision - ACCV 2016: 13th Asian Conference on Computer Vision in Lai, S.H., Lepetit, V., Nishino, K., Sato, Y. (Eds.): ‘A vote-and-verify strategy for fast spatial verification in image retrieval’ (Springer Int. Publishing, Cham, 2017), pp. 321–337.
    20. 20)
      • 28. Tolias, G., Avrithis, Y., Jégou, H.: ‘Image search with selective match kernels: aggregation across single and multiple images’, Int. J. Comput. Vis., 2016, 116, (3), pp. 247–261. Available at http://dx.doi.org/10.1007/s11263-015-0810-4.
    21. 21)
      • 30. Yuan, Y., Fang, J., Wang, Q.: ‘Online anomaly detection in crowd scenes via structure analysis’, IEEE Trans. Cybern., 2015, 45, (3), pp. 548–561.
    22. 22)
      • 16. Li, X., Larson, M., Hanjalic, A.: ‘Pairwise geometric matching for large-scale object retrieval’. 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston, USA, June 2015, pp. 5153–5161.
    23. 23)
      • 5. Sattler, T., Havlena, M., Schindler, K., et al: ‘Large-scale location recognition and the geometric burstiness problem’. 2016 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Las Vegas, USA, June 2016, pp. 1582–1590.
    24. 24)
      • 23. Zheng, L., Yang, Y., Tian, Q.: ‘SIFT meets CNN: a decade survey of instance retrieval’, IEEE Trans. Pattern Anal. Mach. Intell., 2017, 99, (PP), p. 1.
    25. 25)
      • 17. Jégou, H., Douze, M., Schmid, C.: Computer Vision - ECCV 2008: 10th European Conference on Computer Vision in Forsyth, D., Torr, P., Zisserman, A. (Eds.): ‘Hamming embedding and weak geometric consistency for large scale image search’ (Springer, Berlin, Heidelberg, 2008), pp. 304–317.
    26. 26)
      • 11. Gordo, A., Almazán, J., Revaud, J., et al: Computer Vision - ECCV 2016: 14th European Conference in Leibe, B., Matas, J., Sebe, N., Welling, M. (Eds.): ‘Deep image retrieval: learning global representations for image search’ (Springer Int. Publishing, Cham, 2016), pp. 241–257.
    27. 27)
      • 35. Mikulík, A., Perdoch, M., Chum, O., et al: Computer Vision - ECCV 2010: 11th European Conference on Computer Vision in Daniilidis, K., Maragos, P., Paragios, N. (Eds.): ‘Learning a fine vocabulary’ (Springer, Berlin, Heidelberg, 2010), pp. 1–14.
    28. 28)
      • 32. Perdoch, M., Chum, O., Matas, J.: ‘Efficient representation of local geometry for large scale object retrieval’. 2009 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Florida, USA, June 2009, pp. 9–16.
    29. 29)
      • 7. Philbin, J., Chum, O., Isard, M., et al: ‘Object retrieval with large vocabularies and fast spatial matching’. 2007 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Minneapolis, Minnesota, USA, June 2007, pp. 1–8.
    30. 30)
      • 34. Douze, M., Jégou, H.: ‘The Yael library’. Proc. 22nd ACM Int. Conf. on Multimedia (MM'14), New York, NY, USA, 2014, pp. 687–690. Available at http://doi.acm.org/10.1145/2647868.2654892.
    31. 31)
      • 26. Avrithis, Y., Tolias, G.: ‘Hough pyramid matching: speeded-up geometry re-ranking for large scale image retrieval’, Int. J. Comput. Vis., 2014, 107, (1), pp. 1–19. Available at https://doi.org/10.1007/s11263-013-0659-3.
    32. 32)
      • 21. Bay, H., Ess, A., Tuytelaars, T., et al: ‘Speeded-up robust features (SURF)’, Comput. Vis. Image Underst., 2008, 110, (3), pp. 346–359. Available at http://dx.doi.org/10.1016/j.cviu.2007.09.014.
    33. 33)
      • 1. Gammeter, S., Bossard, L., Quack, T., et al: ‘I know what you did last summer: object-level auto-annotation of holiday snaps’. 2009 IEEE 12th Int. Conf. on Computer Vision, Kyoto, Japan, September 2009, pp. 614–621.
    34. 34)
      • 3. Torii, A., Sivic, J., Okutomi, M., et al: ‘Visual place recognition with repetitive structures’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 37, (11), pp. 2346–2359.
    35. 35)
      • 15. Shi, M., Avrithis, Y., Jégou, H.: ‘Early burst detection for memory-efficient image retrieval’. 2015 IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), Boston, USA, June 2015, pp. 605–613.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2016.0504
Loading

Related content

content/journals/10.1049/iet-cvi.2016.0504
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading