Content-based image retrieval (CBIR) systems often incorporate a relevance feedback mechanism in which retrieval is adapted based on users identifying images as relevant or irrelevant. Such relevance decisions are often assumed to be category-based. However, forcing a user to decide upon category membership of an image, even when unfamiliar with a database and irrespective of context, is restrictive. An alternative is to obtain user feedback in the form of relative similarity judgments. The ability of a user to provide meaningful feedback depends on the interface that displays retrieved images and facilitates the feedback. Similarity-based 2D layouts provide context and can enable more efficient visual search. Motivated by these observations, this study describes and evaluates an interactive image browsing and retrieval approach based on relative similarity feedback obtained from 2D image layouts. It incorporates online maximal-margin learning to adapt the image similarity metric used to perform retrieval. A user starts a session by browsing a collection of images displayed in a 2D layout. He/she may choose a query image perceived to be similar to the envisioned target image. A set of images similar to the query are then returned. The user can then provide relational feedback and/or update the query image to obtain a new set of images. Algorithms for CBIR are often characterised empirically by simulating usage based on pre-defined, fixed category labels, deeming retrieved results as relevant if they share a category label with the query. In contrast, the purpose of the system in this study is to enable browsing and retrieval without predefined categories. Therefore evaluation is performed in a target-based setting by quantifying the efficiency with which target images are retrieved given initial queries.

References

1. 1)
  - 19. Faria, F.F., Veloso, A., Almeida, H.M., Valle, E., Torres, R.d.S., Goncalves, M.A., Meira, W.Jr.: ‘Learning to rank for content-based image retrieval’. Int. Conf. on Multimedia Information Retrieval, 2010.
2. 2)
  - 1. Huang, J., Ravi Kumar, S., Mitra, M., Zhu, W., Zabih, R.: ‘Spatial color indexing and applications’, Int. J. Comput. Vis., 1999, 35, pp. 245–268 (doi: 10.1023/A:1008108327226).
3. 3)
  - I.J. Cox , M.L. Miller . The Bayesian image retrieval system, PicHunter: theory, implementation, and psychophysical experiments. IEEE Trans. Image Process. , 1 , 20 - 37
4. 4)
  - 3. Chen, J.-J., Su, C.-R., Grimson, W.E.L., Liu, J.-L., Shiue, D.-H.: ‘Object segmentation of database images by dual multiscale morphological reconstructions and retrieval applications’, IEEE Trans. Image Process., 2012, 21, (2), pp. 828–843 (doi: 10.1109/TIP.2011.2166558).
5. 5)
  - 20. Huang, W., Chan, K.L., Li, H., Lim, J.H., Liu, J., Wong, T.Y.: ‘Content-based medical image retrieval with metric learning via rank correlation’. Int. Workshop on Machine Learning in Medical Imaging, 2010, pp. 18–25.
6. 6)
  - 7. Wang, B., Pan, F., Hu, K., Paul, J.: ‘Manifold-ranking based retrieval using k-regular nearest neighbor graph’, Pattern Recognit., 2012, 45, (4), pp. 1569–1577 (doi: 10.1016/j.patcog.2011.09.006).
7. 7)
  - 21. Wang, G., Forsyth, D., Hoiem, D.: ‘Comparative object similarity for improved recognition with few or no examples’. IEEE Conf. on Computer Vision and Pattern Recognition, 2010.
8. 8)
  - 26. Han, J., McKenna, S.J., Wang, R.: ‘Learning query-dependent distance metrics for interactive image retrieval’. Int. Conf. on Computer Vision Systems, Liege, 2007.
9. 9)
  - 23. Rodden, K.: ‘Evaluating Similarity-Based Visualisations as Interfaces for Image Browsing’, Ph.D. Thesis. University of Cambridge, 2001.
10. 10)
  - 25. Wang, R., McKenna, S.J., Han, J., Ward, A.A.: ‘Visualizing image collections using high-entropy layout distributions’, IEEE Trans. Multimedia, 2010, 12, (8), pp. 803–813 (doi: 10.1109/TMM.2010.2057411).
11. 11)
  - 18. Lee, J.E., Jin, R., Jain, A.K.: ‘Rank-based distance metric learning: An application to image retrieval’. IEEE Conf. on Computer Vision and Pattern Recognition, Anchorage, 2008.
12. 12)
  - 9. Si, L., Jin, R., Hoi, S., Lyu, M.: ‘Collaborative image retrieval via regularized metric learning’, Multimedia Syst., 2006, 12, (1), pp. 34–44 (doi: 10.1007/s00530-006-0033-1).
13. 13)
  - 2. Smith, J.R., Chang, S.F.: ‘Automated binary texture feature sets for image retrieval’. IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Atlanta, 1996.
14. 14)
  - 15. Freund, A., Iyer, R., Schapire, R.E., Lozano-Perez, T.: ‘An efficient boosting algorithm for combining preferences’, J. Mach. Learn. Res., 2003, 4, pp. 933–969.
15. 15)
  - 22. Parikh, D., Grauman, K.: ‘Relative Attributes’. IEEE Int. Conf. on Computer Vision, 2011.
16. 16)
  - 13. Joachims, T.: ‘Optimizing search engines using clickthrough data’. ACM SIGKDD Int. Conf. on Knowledge Discovery and DataMining, Alberta, Canada, 2002.
17. 17)
  - 8. Han, J., Ngan, K., Li, M., Zhang, H.: ‘A memory learning framework for effective image retrieval’, IEEE Trans. Image Process., 2005, 14, (4), pp. 511–524 (doi: 10.1109/TIP.2004.841205).
18. 18)
  - 17. Hu, Y., Li, M., Yu, N.: ‘Multiple-instance ranking: Learning to rank images for image retrieval’. IEEE Conf. on Computer Vision and Pattern Recognition, Anchorage, USA, 2008.
19. 19)
  - 6. Zhang, L., Wang, L., Lin, W.: ‘Semi-supervised biased maximum margin analysis for interactive image retrieval’, IEEE Trans. Image Process., 2012, 21, (4), pp. 2294–2308 (doi: 10.1109/TIP.2011.2177846).
20. 20)
  - 4. Tong, S., Chang, E.: ‘Support vector machine active learning for image retrieval’. ACM Conf. on Multimedia, 2001.
21. 21)
  - 16. Frome, A.: ‘Learning Distance Functions for Examplar-based Object Recognition’, Ph.D. thesis. UC Berkeley, 2007.
22. 22)
  - 14. Schultz, M., Joachims, T.: ‘Learning a distance metric from relative comparisons’. Neural Information Processing Systems, Berlin, 2003.
23. 23)
  - 24. Moghaddam, B., Tian, Q., Lesh, N., Shen, C., Huang, T.S.: ‘Visualization and user-modeling for browsing personal photo libraries’, Int. J. Comput. Vis., 2004, 56, pp. 109–130 (doi: 10.1023/B:VISI.0000004834.62090.74).
24. 24)
  - 5. He, X., Ma, W.Y., Zhang, H.J.: ‘Learning an image manifold for retrieval’. ACM Int. Conf. on Multimedia, New York, 2004.
25. 25)
  - 11. Zhang, L., Wang, L., Lin, W.: ‘Conjunctive patches subspace learning with side information for collaborative image retrieval’, IEEE Trans. Image Process., 2012, 21, (8), pp. 3707–3720 (doi: 10.1109/TIP.2012.2195014).
26. 26)
  - 10. Yang, Y., Nie, F., Xu, D., Luo, J., Zhuang, Y., Pan, Y.: ‘A multimedia retrieval framework based on semi-supervised ranking and relevance feedback’, IEEE Trans. Pattern Anal. Mach. Int., 2012, 34, (4), pp. 723–742 (doi: 10.1109/TPAMI.2011.170).
27. 27)
  - S. Arya , D.M. Mount , N.S. Netanyahu , R. Silverman , A.Y. Wu . An optimal algorithm for approximate nearest neighbor searching fixed dimensions. JACM. , 6 , 891 - 923

Query-dependent metric learning for adaptive, content-based image browsing and retrieval

References

Related content