access icon free Feature fusion method using BoVW framework for enhancing image retrieval

© The Institution of Engineering and Technology
Received 26/04/2018, Accepted 10/06/2019, Published 09/07/2019

The bag-of-visual words (BoVW) has been applied to myriad of recognition problems in computer vision such as object recognition, scene classification and image retrieval due to its scalability and high precision. However, their performance is subservient in certain datasets, especially in natural image datasets, mainly due to the lack of consideration of image cues such as colour, texture etc. which are not prime features while computing invariant descriptors, on which BoVW models are generally built on. Hence, this study describes a multi-cue fusion approach for BoVW framework, exploiting both early and late fusion methods, to improve the retrieval performance, mainly in natural image datasets. For this, a composite edge and colour descriptor is proposed to describe the local regions of the image along with the invariant feature descriptor Speeded Up Robust Features (SURF). Independent vocabularies are built based on these descriptors and images in the dataset are encoded to form two histograms using the respective vocabularies. The histograms are further fused to characterise the image. The retrieval is carried out by matching the histograms. Experimental results show that significant increment in the average precision can be attained by combining the proposed descriptor with invariant descriptors.

Inspec keywords: image retrieval; image fusion; image matching; computer vision; edge detection; image colour analysis; image enhancement; image classification; object recognition; feature extraction

Other keywords: image retrieval; scene classification; composite edge descriptor; multicue fusion approach; late fusion methods; recognition problems; computer vision; feature fusion method; invariant feature descriptor SURF; retrieval performance; natural image datasets; early fusion methods; BoVW framework; bag-of-visual words; invariant descriptors; composite colour descriptor; object recognition; BoVW models

Subjects: Computer vision and image processing techniques; Sensor fusion; Image recognition; Information retrieval techniques

References

    1. 1)
      • 6. Vigo, D.A.R., Khan, F.S., Van De Weijer, J., et al: ‘The impact of color on bag-of-words based object recognition’. 2010 20th Int. Conf. Pattern Recognition (ICPR), Istanbul, Turkey, August 2010, pp. 1549–1553.
    2. 2)
      • 23. Yang, J., Yu, K., Gong, Y., et al: ‘Linear spatial pyramid matching using sparse coding for image classification’. IEEE Conf. Computer Vision and Pattern Recognition, 2009, Miami, USA, June 2009, pp. 1794–1801.
    3. 3)
      • 18. Vedaldi, A., Zisserman, A.: ‘Efficient additive kernels via explicit feature maps’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (3), pp. 480–492.
    4. 4)
      • 1. Arandjelović, R., Zisserman, A.: ‘Three things everyone should know to improve object retrieval’. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Rhode Island, USA, 2012, pp. 2911–2918.
    5. 5)
      • 8. Abdel-Hakim, A.E., Farag, A.A.: ‘CSIFT: a SIFT descriptor with color invariant characteristics’. IEEE Computer Society Conf. Computer Vision and Pattern Recognition (CVPR'06), New York, USA, 2006vol. 2, pp. 1978–1983.
    6. 6)
      • 11. Yu, J., Qin, Z., Wan, T., et al: ‘Feature integration analysis of bag-of-features model for image retrieval’, Neurocomputing, 2013, 120, pp. 355–364.
    7. 7)
      • 16. Murala, S., Maheshwari, R.P., Balasubramanian, R.: ‘Directional local extrema patterns: a new descriptor for content based image retrieval’, Int. J. Multimed. Inf. Retr., 2012, 1, (3), pp. 191–203.
    8. 8)
      • 17. Murala, S., Wu, Q.J.: ‘Expert content-based image retrieval system using robust local patterns’, J. Vis. Commun. Image Represent., 2014, 25, (6), pp. 1324–1334.
    9. 9)
      • 24. Douze, M., Jégou, H., Harsimrat, S., et al: ‘Evaluation of GIST descriptors for web-scale image search’. Conf. Image and Video Retrieval, Santorini, Greece, 2009.
    10. 10)
      • 19. Available at http://wang.ist.psu.edu/docs/related/, accessed on 24th November 2017.
    11. 11)
      • 26. Jégou, H., Perronnin, F., Douze, M., et al: ‘Aggregating local image descriptors into compact codes’, EEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (9), pp. 1704–1716.
    12. 12)
      • 5. Bosch, A., Zisserman, A., Muñoz, X.: ‘Scene classification using a hybrid generative/discriminative approach’, IEEE Trans. Pattern Anal. Mach. Intell., 2008, 30, (4), pp. 712–727.
    13. 13)
      • 2. Vedaldi, A., Gulshan, V., Varma, M., et al: ‘Multiple kernels for object detection’. 2009 IEEE 12th Int. Conf. Computer Vision, Kyoto, Japan, September 2009.
    14. 14)
      • 12. Wengert, C., Douze, M., Jégou, H.: ‘Bag-of-colors for improved image search’. Proc. 19th ACM Int. Conf. Multimedia, Scottsdale, USA, November 2011, pp. 1437–1440.
    15. 15)
      • 20. Available at http://www.ci.gxnu.edu.cn/cbir/Dataset.aspx, accessed on 24th November 2017.
    16. 16)
      • 9. Chu, D.M., Smeulders, A.W.: ‘Color invariant SURF in discriminative object tracking’. European Conf. Computer Vision, Berlin, Heidelberg, September 2010, pp. 62–75.
    17. 17)
      • 10. Fu, J., Jing, X., Sun, S., et al: ‘C-SURF: colored speeded up robust features’. Int. Conf. Trustworthy Computing and Services, Berlin, Heidelberg, May 2012, pp. 203–210.
    18. 18)
      • 22. Jurie, F., Triggs, B.: ‘Creating efficient codebooks for visual recognition’. Tenth IEEE Int. Conf. Computer Vision (ICCV'05), Beijing, China, October 2005, vol. 1, pp. 604–610.
    19. 19)
      • 15. Walia, E., Pal, A.: ‘Fusion framework for effective color image retrieval’, J. Vis. Commun. Image Represent., 2014, 25, (6), pp. 1335–1348.
    20. 20)
      • 3. Lowe, D.G.: ‘Distinctive image features from scale-invariant keypoints’, Int. J. Comput. Vis., 2004, 60, (2), pp. 91–110.
    21. 21)
      • 13. Khan, F.S., Van de Weijer, J., Vanrell, M.: ‘Modulating shape features by color attention for object recognition’, Int. J. Comput. Vis., 2012, 98, (1), pp. 49–64.
    22. 22)
      • 7. Fan, P., Men, A., Chen, M., et al: ‘Color-SURF: a surf descriptor with local kernel color histograms’. 2009 IEEE Int. Conf. Network Infrastructure and Digital Content, Beijing, China, November 2009, pp. 726–730.
    23. 23)
      • 4. Bay, H., Ess, A., Tuytelaars, T., et al: ‘Speeded-up robust features (SURF)’, Comput. Vis. Image Underst., 2008, 110, (3), pp. 346–359.
    24. 24)
      • 21. Jegou, H., Douze, M., Schmid, C.: ‘Hamming embedding and weak geometric consistency for large scale image search’. European Conf. Computer Vision, Marseille, France, 12–18 October 2008, vol. 1, pp. 304–317.
    25. 25)
      • 25. Douze, M., Ramisa, A., Schmid, C.: ‘Combining attributes and Fischer vectors for efficient image retrieval’. Computer Vision and Pattern Recognition, Colorado Springs, USA, 2011.
    26. 26)
      • 14. Zhang, S., Yang, M., Cour, T., et al: ‘Query specific rank fusion for image retrieval’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 37, (4), pp. 803–815.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-ipr.2018.5381
Loading

Related content

content/journals/10.1049/iet-ipr.2018.5381
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading