Visual tracking is a popular and challenging topic in computer vision and robotics. Owing to changes in the appearance of the target and complicated variations that may occur in various scenes, online learning scheme is necessary for advanced visual tracking framework to adopt. This paper briefly introduces the challenges and applications of visual tracking and focuses on discussing the state-of-the-art online-learning-based tracking methods by category. We provide detail descriptions of representative methods in each category, and examine their pros and cons. Moreover, several most representative algorithms are implemented to provide quantitative reference. At last, we outline several trends for future visual tracking research.

References

1. 1)
  - 61. Jiang, N., Liu, W.Y., Wu, Y.: ‘Learning adaptive metric for robust visual tracking’, IEEE Trans. Imaging Process., 2011, 20, (8), pp. 200–214.
2. 2)
  - 47. Bai, Y.C., Tang, M.: ‘Robust tracking via weakly supervised ranking SVM’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, USA, June 2012, pp. 1855–1861.
3. 3)
  - 7. Chiverton, J., Mirmehdi, M., Xie, X.H.: ‘On-line learning of shape information for object segmentation and tracking’. Proc. British Machine Vision Conf. (BMVC), London, UK, September 2009, pp. 1–11.
4. 4)
  - 11. Kwon, J., Lee, K.M.: ‘Tracking by sampling trackers’. Proc. IEEE Int. Conf. Computer Vision (ICCV), Barcelona, Spanish, November 2011, pp. 1195–1202.
5. 5)
  - 52. Santner, J., Leistner, C., Saffari, A., Pock, T., Bischof, H.: ‘PROST: parallel robust online simple tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010, pp. 1–8.
6. 6)
  - 42. Zeisl, B., Leistner, C., Saffari, A., Bischof, H.: ‘On-line semi-supervised multiple-instance boosting’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010, pp. 1879–1886.
7. 7)
  - 38. Grabner, H., Bischof, H.: ‘Real-time tracking via on-line boosting’. Proc. British Machine Vision Conf. (BMVC), Edinburgh, England, September 2006, pp. 47–56.
8. 8)
  - 30. Lu, X.Q., Yuan, Y., Yan, P.K.: ‘Robust visual tracking with discriminative sparse learning’, Pattern Recognit., 2013, 46, (7), pp. 1762–1771 (doi: 10.1016/j.patcog.2012.11.016).
9. 9)
  - 33. Nguyen, H.T., Smeulders, A.: ‘Tracking aspects of the foreground against the background’. Proc. European Conf. Computer Vision (ECCV), Prague, Czech Republic, May 2004, pp. 446–456.
10. 10)
  - 19. Wu, Y., Cheng, J., Wang, J., Lu, H.: ‘Real-time visual tracking via incremental covariance tensor learning’. Proc. IEEE Int. Conf. Computer Vision (ICCV), Kyoto, Japan, September 2009, pp. 1631–1638.
11. 11)
  - 53. Yao, R., Shi, Q.F., Shen, C.H.: ‘Robust tracking with weighted online structured learning’. Proc. European Conf. Computer Vision (ECCV), Firenze, Italy, October 2012, pp. 158–172.
12. 12)
  - 34. Wang, J., Chen, X., Gao, W.: ‘Online selecting discriminative tracking features using particle filter’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, June 2005, pp. 1037–1042.
13. 13)
  - 69. Yu, Q., Dinh, T.B., Medioni, G.: ‘Online tracking and reacquisition using co-trained generative and discriminative trackers’. Proc. European Conf. Computer Vision (ECCV), Marseille, France, October 2008, pp. 678–691.
14. 14)
  - 75. Yang, M., Wu, Y., Lao, S.: ‘Intelligent collaborative tracking by mining auxiliary objects’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), New York, NY, USA, June 2006, pp. 697–704.
15. 15)
  - 35. Li, G., Liang, D., Huang, Q., Jiang, S.Q., Gao, W.: ‘Object tracking using incremental 2D-LDA learning and Bayes inference’. Proc. IEEE Int. Conf. Image Processing (ICIP), San Diego, California, USA, October 2008, pp. 1568–1571.
16. 16)
  - 45. Sankaranarayanan, K., Davis, J.W.: ‘One-class multiple instance learning and applications to target tracking’. Proc. Asian Conf. Computer Vision (ACCV), Daejeon, Korea, November 2012, pp. 1–14.
17. 17)
  - 5. Yang, H.X., Song, Z., Chen, R.N.: ‘An incremental PCA-HOG descriptor for robust visual hand tracking’. Proc. Int. Symp. Visual Computing (ISVC), Las Vegas, Nevada, USA, November 2010, pp. 687–695.
18. 18)
  - 54. Li, Y., Ai, H.Z., Yamashita, T., Lao, S.H., Kawade, M.: ‘Tracking in low frame rate video: a cascade particle filter with discriminative observers of different life spans’, IEEE Trans. Pattern Anal. Mach. Intell., 2008, 30, (10), pp. 1728–1740 (doi: 10.1109/TPAMI.2008.73).
19. 19)
  - 60. Wang, X.Y., Hua, G., Han, T.X.: ‘Discriminative tracking by metric learning’. Proc. European Conf. Computer Vision (ECCV), Grete, Greece, September 2010, pp. 200–214.
20. 20)
  - 29. Jia, X., Lu, H., Yang, M.H.: ‘Visual tracking via adaptive structural local sparse appearance model’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, USA, June 2012, pp. 1822–1829.
21. 21)
  - 15. Lee, K., Kriegman, D.: ‘Online learning of probabilistic appearance manifolds for video-based recognition and tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Diego, USA, June 2005, pp. 852–859.
22. 22)
  - 17. Wen, J., Gao, X.: ‘Incremental learning of weighted tensor subspace for visual tracking’. Proc. IEEE Int. Conf. Systems, Man and Cybernetics (SMC), San Antonio, TX, USA, October 2009, pp. 3688–3693.
23. 23)
  - 62. Jiang, N., Liu, W.Y., Wu, Y.: ‘Order determination and sparsity-regularized metric learning for adaptive visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, USA, June 2012, pp. 1956–1963.
24. 24)
  - 66. Lin, R.S., Ross, D., Lim, J., Yang, M.H.: ‘Adaptive discriminative generative model and its applications’, Adv. Neural Inf. Process. Syst., 2004, 17, pp. 801–808.
25. 25)
  - 13. Lim, J., Ross, D., Lin, R.S., Yang, M.H.: ‘Incremental learning for visual tracking’, in Weiss, Y., Bottou, L., (Eds.), ‘Advances in neural information processing systems’ (MTI Press, 2005), pp. 793–800.
26. 26)
  - 55. Stalder, S., Grabner, H., Gool, L.V.: ‘Dynamic objectness for adaptive tracking’. Proc. Asian Conf. Computer Vision (ACCV), Daejeon, Korea, 2012, pp. 43–56.
27. 27)
  - D. Ross , J. Lim , R. Lin , M.H. Yang . Incremental learning for robust visual tracking. Int. J. Comput. Vis. , 125 - 141
28. 28)
  - 72. Aeschliman, C., Park, J., Kak, A.C.: ‘A probabilistic framework for joint segmentation and tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010, pp. 1371–1378.
29. 29)
  - 77. Liu, Z., Shen, H., Feng, G.Y., Hua, D.W.: ‘Tracking objects using shape context matching’, Neurocomputing, 2012, 83, pp. 47–55 (doi: 10.1016/j.neucom.2011.11.012).
30. 30)
  - 18. Li, X., Hu, W., Zhang, Z., Zhang, X., Luo, G.: ‘Visual tracking via incremental log-Euclidean Riemannian subspace learning’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, USA, June 2008, pp. 1–8.
31. 31)
  - 24. Li, H.X., Shen, C.H., Shi, Q.F.: ‘Real-time visual tracking using compressive sensing’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA, June 2011, pp. 1305–1312.
32. 32)
  - 59. Kalal, Z., Matas, J., Mikolajczyk, K.: ‘P-N learning: boot-strapping binary classifiers by structural constraints’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010, pp. 49–56.
33. 33)
  - 20. Lu, K., Ding, Z.M., Ge, S.: ‘Locally connected graph for visual tracking’. Neurocomputing, 2013, 120, pp. 45–53.
34. 34)
  - 56. Yang, M., Yuan, J., Wu, Y.: ‘Spatial selection for attentional visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Minneapolis, Minnesota, USA, June 2007, pp. 1–8.
35. 35)
  - 74. Zhang, S.P., Ya, H.X., Zhou, H.Y., Sun, X., Liu, S.H.: ‘Robust visual tracking based on online learning sparse representation’, Neurocomputing, 2013, 100, pp. 31–40 (doi: 10.1016/j.neucom.2011.11.031).
36. 36)
  - 64. Li, X., Shen, C.H., Shi, Q.F., Dick, A., Hengel, A.V.: ‘Non-sparse linear representations for visual tracking with online reservoir metric learning’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, USA, June 2012, pp. 1760–1767.
37. 37)
  - 68. Zhang, X.Q., Hu, W.M., Maybank, S., Li, X.: ‘Graph based discriminative learning for robust and efficient object tracking’. Proc. IEEE Int. Conf. Computer Vision (ICCV), Rio de Janeiro, Brazil, October 2007, pp. 1–8.
38. 38)
  - 37. Grabner, H., Bischof, H.: ‘On-line boosting and vision’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), New York, NY, USA, June 2006, pp. 260–267.
39. 39)
  - 41. Babenko, B., Yang, M.H., Belongie, S.: ‘Visual tracking with online multiple instance learning’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Miami, Florida, USA, June 2009, pp. 983–990.
40. 40)
  - R.T. Collins , Y. Liu , M. Leordeanu . Online selection of discriminative tracking features. IEEE Trans. Pattern Anal. Mach. Intell. , 10 , 1631 - 1643
41. 41)
  - 27. Chen, F., Wang, Q., Wang, S., Zhang, W.D., Xu, W.L.: ‘Object tracking via appearance modeling and sparse representation’, Int. J. Image Vis. Comput., 2011, 29, pp. 787–796 (doi: 10.1016/j.imavis.2011.08.006).
42. 42)
  - 9. Xu, Y.L., Zhou, H.F., Wang, Q., Lin, L.: ‘Real time object of interest tracking by learning composite patch-based templates’. Proc. IEEE Int. Conf. Image Processing (ICIP), Orlando, FL, USA, September 2012, pp. 389–392.
43. 43)
  - 58. Kalal, Z., Matas, J., Mikolajczyk, K.: ‘Online learning of robust object detectors during unstable tracking’. Proc. IEEE Int. Conf. Computer Vision Workshop (ICCV Workshop), Kyoto, Japan, October 2009, pp. 1417–1424.
44. 44)
  - 4. Fussenegger, M., Roth, P., Bischof, H., Deriche, R., Pinz, A.: ‘A level set framework using a new incremental, robust active shape model for object segmentation and tracking’, Image Vis. Comput., 2009, 27, (8), pp. 1157–1168 (doi: 10.1016/j.imavis.2008.10.014).
45. 45)
  - 65. Cong, Y., Yuan, J.S., Tang, Y.D.: ‘Object tracking via online metric learning’. Proc. IEEE Int. Conf. Image Processing (ICIP), Orlando, Florida, USA, September 2012, pp. 417–420.
46. 46)
  - 28. Bai, T.X., Li, Y.F.: ‘Robust visual tracking with structured sparse representation appearance model’, Pattern Recognit., 2012, 45, (6), pp. 2390–2404 (doi: 10.1016/j.patcog.2011.12.004).
47. 47)
  - 73. Wang, Q., Chen, F., Xu, W.L., Yang, M.: ‘Online discriminative object tracking with local sparse representation’. Proc. IEEE Workshop on Applications of Computer Vision (WACV), Clearwater, Florida, USA, January 2012, pp. 425–432.
48. 48)
  - 49. Saffari, A., Leistner, C., Santner, J., Godec, M., Bischof, H.: ‘On-line random forests’. Proc. IEEE Int. Conf. Computer Vision Workshop (ICCV Workshop), Kyoto, Japan, October 2009, pp. 1393–1400.
49. 49)
  - 16. Li, X., Hu, W.M., Zhang, Z.F.: ‘Robust visual tracking based on incremental tensor subspace learning’. Proc. IEEE Int. Conf. Computer Vision (ICCV), Rio de Janeiro, Brazil, October 2007, pp. 1–8.
50. 50)
  - 39. Leistner, C., Granber, H., Bischof, H.: ‘Semi-supervised boosting using visual similarity learning’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, USA, June 2008, pp. 1–8.
51. 51)
  - 22. Mei, X., Ling, H.B.: ‘Robust visual tracking using l1 minimization’. Proc. IEEE Int. Conf. Computer Vision (ICCV), Kyoto, Japan, September 2009, pp. 1436–1443.
52. 52)
  - S. Kumar , M. Hebert . Discriminative random fields. Int. J. Comput. Vis. , 2 , 179 - 202
53. 53)
  - 36. Avidan, S.: ‘Ensemble tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Diego, CA, USA, June 2005, pp. 494–501.
54. 54)
  - 63. Tsagkatakis, G., Savakis, A.: ‘Online distance metric learning for object tracking’, IEEE Trans. Circuits Syst. Video Technol., 2011, 21, (12), pp. 1810–1821 (doi: 10.1109/TCSVT.2011.2133970).
55. 55)
  - 10. Kwon, J., Lee, K.M.: ‘Visual tracking decomposition’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010, pp. 1269–1276.
56. 56)
  - 26. Liu, R., Huang, J.Z., Yang, L., Kulikowsk, C.A.: ‘Robust tracking using local sparse appearance model and K-selection’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA, June 2011, pp. 1313–1320.
57. 57)
  - 67. Woodley, T., Stenger, B., Cipolla, R.: ‘Tracking using online feature selection and a local generative model’. Proc. British Machine Vision Conf. (BMVC), UK, September 2007, pp. 790–799.
58. 58)
  - 70. Yin, Z., Collins, R.T.: ‘Shape constrained figure-ground segmentation and tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Miami, Florida, USA, June 2009, pp. 731–738.
59. 59)
  - 48. Gu, S., Zheng, Y., Tomasi, C.: ‘Efficient visual object tracking with online nearest neighbor classifier’. Proc. Asian Conf. Computer Vision (ACCV), Queenstown, New Zealand, November 2011, pp. 271–282.
60. 60)
  - 50. Petrovic, N., Jovanov, L., Pizurica, A., Philips, W.: ‘Object tracking using naive bayesian classifiers’. Proc. Int. Conf. Advanced Concepts for Intelligent Vision, Juan-les-Pins, France, October 2008, pp. 775–784.
61. 61)
  - 78. Barranco, F., Díaz, J., Ros, E., Del Pino, B.: ‘Visual system based on artificial retina for motion detection’, IEEE Trans. Syst. Man Cybern. B, Cybern., 2009, 39, (3), pp. 752–762 (doi: 10.1109/TSMCB.2008.2009067).
62. 62)
  - X. Mei , H. Ling . Robust visual tracking and vehicle classification via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. , 11 , 2259 - 2272
63. 63)
  - 40. Grabner, H., Leistner, C., Bischof, H.: ‘Semi-supervised on-line boosting for robust tracking’. Proc. European Conf. Computer Vision (ECCV), Marseille, France, October 2008, pp. 234–247.
64. 64)
  - 43. Li, W., Duan, L.X., Tsang, I.W., Xu, D.: ‘Batch mode adaptive multiple instance learning for computer vision tasks’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Providence, Rhode Island, USA, June 2012, pp. 2368–2375.
65. 65)
  - 57. Kalal, Z., Mikolajczyk, K., Matas, J.: ‘Tracking-learning-detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2012, 34, (7), pp. 1409–1422 (doi: 10.1109/TPAMI.2011.239).
66. 66)
  - I. Matthews , T. Ishikawa , S. Baker . The template update problem. IEEE Trans. Pattern Anal. Mach. Intell. , 6 , 810 - 815
67. 67)
  - 3. Tu, J.L., Tao, H.: ‘Online updating appearance generative mixture model for meanshift tracking’. Proc. Asian. Conf. Computer Vision (ACCV), Hyderabad, India, January 2006, pp. 694–703.
68. 68)
  - 76. Grabner, H., Matas, J., Gool, L.V., Cattin, P.: ‘Tracking the invisible: learning where the object might be’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), San Francisco, CA, USA, June 2010, pp. 1285–1292.
69. 69)
  - 8. Liu, X.B., Lin, L., Yan, S.C., Jin, H., Jiang, W.B.: ‘Adaptive object tracking by learning hybrid template online’, IEEE Trans. Circuits Syst. Video Technol., 2011, 21, (11), pp. 1588–1599 (doi: 10.1109/TCSVT.2011.2129410).
70. 70)
  - 25. Liu, B., Yang, L., Huang, J., Meer, P., Gong, L., Kulikowski, C.A.: ‘Robust and fast collaborative tracking with two stage sparse optimization’. Proc. European Conf. Computer Vision (ECCV), Grete, Greece, September 2010, pp. 624–637.
71. 71)
  - 6. Chiverton, J., Xie, X.H.: ‘Automatic bootstrapping and tracking of object contours’, IEEE Trans. Image Process., 2012, 21, (3), pp. 1231–1245 (doi: 10.1109/TIP.2011.2167343).
72. 72)
  - S.K. Zhou , R. Chellappa , B. Moghaddam . Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. Image Process. , 1491 - 1506
73. 73)
  - 31. Stern, H., Efros, B.: ‘Adaptive color space switching for face tracking in multi-colored lighting environments’. Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, Washington, DC, USA, May 2002, pp. 236–241.
74. 74)
  - 51. Zhang, K.H., Zhang, L., Yang, M.: ‘Real-time compressive Tracking’. Proc. European Conf. Computer Vision (ECCV), Firenze, Italy, October 2012, pp. 864–877.
75. 75)
  - 1. Jepson, A.D., Fleet, D.J., El-Maraghi, T.F.: ‘Robust online appearance models for visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Kauai, HI, USA, December 2001, pp. 415–422.
76. 76)
  - 12. Ross, D., Lim, J., Yang, M.H.: ‘Adaptive probabilistic visual tracking with incremental subspace update’. Proc. European Conf. Computer Vision (ECCV), Prague, Czech Republic, May 2004, pp. 470–482.
77. 77)
  - 44. Parag, T., Porikli, F., Elgammal, A.: ‘Boosting adaptive linear weak classifiers for online learning and tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), Anchorage, Alaska, USA, June 2008, pp. 1–8.
78. 78)
  - 46. Tian, M., Zhang, W., Liu, F.: ‘On-line ensemble SVM for robust object tracking’. Proc. Asian Conf. Computer Vision (ACCV), Tokyo, Japan, November 2007, pp. 355–364.

Survey of single-target visual tracking methods based on online learning

References

Related content