access icon openaccess Learning a unified tracking-and-detection framework with distractor-aware constraint for visual object tracking

This is an open access article published by the IET under the Creative Commons Attribution -NonCommercial License (http://creativecommons.org/licenses/by-nc/3.0/)
Received 14/10/2019, Accepted 18/06/2020, Revised 18/06/2020, Published 08/07/2020

Most of the correlation filter (CF)-based trackers utilise the circulant structure of the training samples to learn discriminative filters to identify the tracked target, which has shown excellent performance in terms of both tracking accuracy and speed. However, CF-based trackers possess two potential drawbacks: the search regions are limited to the small local neighbourhood for the high-speed tracking purpose; thus, they usually have very few context information and tend to drift from a target in extreme attributes, e.g. background clutter, large-scale variation, and fast motion. Another is that once the tracking target is lost under large displacement motion, it cannot be re-identified in subsequent frames. In this study, the authors propose a unified tracking-and-detection framework with distractor-aware (UTDF-DA), which involves both context learning and target re-identification with a target-aware detector to solve the drawbacks mentioned above. They first incorporate the distractor constraint as context knowledge into a continuous correlation filter for distractor-aware filter learning. Then a single-shot multibox detector-based target-aware detector is trained by domain-specific meta-training approach for deep detection features and hard-negative samples generation. Moreover, they propose the spatial-scale consistency verification method for the target re-identification task. Compared with existing state-of-the-art trackers, UTDF-DA (the authors’) tracker can achieve improved tracking performance in terms of both accuracy and robustness; they demonstrate its effectiveness and efficiency with comprehensive experiments on OTB-2015, VOT-2016, and VOT-2017 benchmarks.

Inspec keywords: learning (artificial intelligence); feature extraction; target tracking; object detection; object tracking

Other keywords: continuous correlation filter; CF-based trackers; visual object tracking; single-shot multibox detector-based target-aware detector; correlation filter-based trackers; unified tracking-and-detection framework; context learning; distractor-aware filter learning; distractor-aware constraint; tracking target; target re-identification; domain-specific meta-training

Subjects: Knowledge engineering techniques; Computer vision and image processing techniques; Optical, image and video signal processing

References

    1. 1)
      • 4. Danelljan, M., Khan, F., Felsberg, M., et al: ‘Learning spatially regularized correlation filters for visual tracking’. Proc. IEEE Int. Conf. Computer Vision Workshop (ICCV), 2015, pp. 4310–4318.
    2. 2)
      • 6. Danelljan, M., Khan, F., Felsberg, M., et al: ‘Convolutional features for correlation filter based visual tracking’. Proc. IEEE Int. Conf. Computer Vision Workshop (ICCV), 2015, pp. 621–629.
    3. 3)
      • 16. Dyer, S.A., Dyer, J.S.: ‘Cubic-spline interpolation: part 1’, IEEE Instrum. Meas. Mag., 2001, 4, (1), pp. 44–46.
    4. 4)
      • 35. Kristan, M., Leonardis, A., Matas, J., et al: ‘The visual object tracking VOT2017 challenge results’. Proc. IEEE Int. Conf. Computer Vision Workshops (ICCVW), 2017, pp. 1949–1972.
    5. 5)
      • 23. Kingma, D.P., Ba, J.: ‘Adam: a method for stochastic optimization’. CoRR, abs/1412.6980, 2014.
    6. 6)
      • 20. Park, E., Berg, A.C.: ‘Meta-tracker: fast and robust online adaptation for visual object trackers’. Proc. European Conf. Computer Vision (ECCV), 2018, pp. 587–604.
    7. 7)
      • 27. Ran, T., Efstratios, G., Arnold, S.W.M.: ‘Siamese instance search for tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1420–1429.
    8. 8)
      • 13. Wang, M., Liu, Y., Huang, Z.Y.: ‘Large margin object tracking with circulant feature maps’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2017, pp. 4021–4029.
    9. 9)
      • 17. Wei, L., Dragomir, A., Dumitru, E., et al: ‘SSD: single shot MultiBox detector’. Proc. European Conf. Computer Vision (ECCV), 2016, pp. 21–37.
    10. 10)
      • 8. Danelljan, M., Bhat, G., Khan, F., et al: ‘ECO: efficient convolution operators for tracking’. Proc. European Conf. Computer Vision (ECCV), 2017, pp. 6931–6939.
    11. 11)
      • 26. Hare, S., Saffari, A., Torr, H.S.: ‘Struck: structured output tracking with kernels’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (10), pp. 2096–2109.
    12. 12)
      • 25. Zhang, J.M., Ma, S.G., Stan, S.: ‘MEEM: robust tracking via multiple experts using entropy minimization’. Proc. European Conf. Computer Vision (ECCV), 2014, pp. 188–203.
    13. 13)
      • 32. Danelljan, M., Khan, F., Felsberg, M., et al: ‘Adaptive decontamination of the training set: A unified formulation for discriminative visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1430–1438.
    14. 14)
      • 11. Simonyan, K., Zisserman, A.: ‘Very deep convolutional networks for large-scale image recognition’. Proc. CoRR, abs/1409.1556, 2014.
    15. 15)
      • 7. Danelljan, M., Khan, F., Felsberg, M., et al: ‘Beyond correlation filters: learning continuous convolution operators for visual tracking’. Proc. European Conf. Computer Vision (ECCV), 2016, pp. 472–488.
    16. 16)
      • 34. Kristan, M., Leonardis, A., Matas, J., et al: ‘The visual object tracking VOT2016 challenge results’. Proc. European Conf. Computer Vision Workshops (ECCVW), 2016, pp. 777–823.
    17. 17)
      • 31. Qi, Y., Zhang, S., Qin, L., et al: ‘Hedged deep tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 4303–4311.
    18. 18)
      • 33. Henriques, J., Caseiro, R., Martins, P., et al: ‘Staple: complementary learners for real-time tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 1401–1409.
    19. 19)
      • 5. Zhang, M.D., Wang, Q., Xing, J.L., et al: ‘Visual tracking via spatially aligned correlation filters network’. Proc. Euro. Conf. Computer Vision (ECCV), 2018, pp. 472–488.
    20. 20)
      • 29. Luca, B., Jack, V., Andrea, V., et al: ‘Fully-convolutional siamese networks for object tracking’. Proc. European Conf. Computer Vision (ECCV), 2016, pp. 850–865.
    21. 21)
      • 30. Hyeonseob, N., Bohyung, H.: ‘Learning multi-domain convolutional neural networks for visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 429–4302.
    22. 22)
      • 36. Kristan, M., Leonardis, A., Matas, J., et al: ‘A novel performance evaluation methodology for single-target trackers’, IEEE Trans. Pattern Anal. Mach. Intell., 2016, 38, (11), pp. 2137–2155.
    23. 23)
      • 28. Fan, H., Ling, H.: ‘SANet: structure-aware network for visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops (CVPR), 2017, pp. 2217–2224.
    24. 24)
      • 19. Luca, B., Jo, H., Jack, V., et al: ‘Learning feed-forward one-shot learners’. Proc. Adv. Neural Inf. Processing System (NIPS), 2016, pp. 523–531.
    25. 25)
      • 3. Danelljan, M., Khan, F., Felsberg, M., et al: ‘Adaptive color attributes for real-time visual tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2018, pp. 484–500.
    26. 26)
      • 18. Russakovsky, O., Deng, J., Su, H., et al: ‘Imagenet large scale visual recognition challenge’, Int. J. Comput. Vis. (IJCV), 2015, 115, (3), pp. 211–252.
    27. 27)
      • 21. Finn, C., Abbeel, P., Levine, S.: ‘Model-agnostic meta-learning for fast adaptation of deep networks’. Proc. Int. Conf. Machine Learning (ICML), 2017, pp. 1126–1135.
    28. 28)
      • 24. Kalal, Z., Mikolajczyk, K., Matas, J.: ‘Tracking-learning-detection’, IEEE Trans. Pattern Anal. Mach. Intell., 2010, 34, (1), pp. 1409–1422.
    29. 29)
      • 22. Sachin, R., Hugo, L.: ‘Optimization as a model for few-shot learning’. Proc. Int. Conf. Learning Representations (ICLR), 2017.
    30. 30)
      • 10. Henriques, J., Caseiro, R., Martins, P., et al: ‘Highspeed tracking with kernelized correlation filters’, IEEE Trans. Pattern Anal. Mach. Intell., 2015, 37, (3), pp. 583–596.
    31. 31)
      • 14. Matthias, M., Neil, S., Bernard, G.: ‘Context-aware correlation filter tracking’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1387–1395.
    32. 32)
      • 9. Henriques, J., Caseiro, R., Martins, P., et al: ‘Exploiting the circulant structure of tracking-by-detection with kernels’. Proc: the European Conf. Computer Vision (ECCV), 2012, pp. 702–715.
    33. 33)
      • 15. Wu, Y., Lim, J.W., Yang, M.H.: ‘Online object tracking: A benchmark’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2013, pp. 2411–2418.
    34. 34)
      • 12. Zhu, G., Porikli, F., Li, H.: ‘Beyond local search: tracking objects everywhere with instance-specific proposals’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2016, pp. 943–951.
    35. 35)
      • 2. Tang, M., Yu, B., Zhang, F., et al: ‘High-speed tracking with multi-kernel correlation filters’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2018, pp. 4874–4883.
    36. 36)
      • 1. Bolme, D.S., Beveridge, J.R., Draper, B.A., et al: ‘Visual object tracking using adaptive correlation filters’. Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2010, pp. 2544–2550.
http://iet.metastore.ingenta.com/content/journals/10.1049/joe.2019.1205
Loading

Related content

content/journals/10.1049/joe.2019.1205
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading