Real-time text tracking in natural scenes

Carlos Merino-Gracia; Majid Mirmehdi

Real-time text tracking in natural scenes

View Fulltext

Author(s): Carlos Merino-Gracia ^{1, 2} and Majid Mirmehdi ²
- Affiliations: 1: Neurochemistry and Neuroimaging Laboratory, University of La Laguna, La Laguna, Spain;
  2: Visual Information Laboratory, University of Bristol, Bristol, UK
Source: Volume 8, Issue 6, December 2014, p. 670 – 681
DOI: 10.1049/iet-cvi.2013.0217 , Print ISSN 1751-9632, Online ISSN 1751-9640

Received 17/08/2013, Accepted 28/03/2014, Revised 24/03/2014, Published 16/06/2014

The authors present a system that automatically detects, recognises and tracks text in natural scenes in real-time. The focus of the author's method is on large text found in outdoor environments, such as shop signs, street names, billboards and so on. Built on top of their previously developed techniques for scene text detection and orientation estimation, the main contribution of this work is to present a complete end-to-end scene text reading system based on text tracking. They propose to use a set of unscented Kalman filters to maintain each text region's identity and to continuously track the homography transformation of the text into a fronto-parallel view, thereby being resilient to erratic camera motion and wide baseline changes in orientation. The system is designed for continuous, unsupervised operation in a handheld or wearable system over long periods of time. It is completely automatic and features quick failure recovery and interactive text reading. It is also highly parallelised to maximise usage of available processing power and achieve real-time operation. They demonstrate the performance of the system on sequences recorded in outdoor scenarios.

References

1. 1)
  - 29. Pilu, M.: ‘Extraction of illusory linear clues in perspectively skewed documents’. Computer Vision and Pattern Recognition, 2001, pp. 363–368.
2. 2)
  - 10. Kasturi, R., Goldgof, D., Soundararajan, P., Manohar, V., Garofolo, J., Bowers, R., Boonstra, M., Korzhova, V., Zhang, J.: ‘Framework for performance evaluation of face, text, and vehicle detection and tracking in video: data, metrics, and protocol’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (2), pp. 319–336 (doi: 10.1109/TPAMI.2008.57).
3. 3)
  - 25. González, A., Bergasa, L.M.: ‘A text reading algorithm for natural images’, Image Vis. Comput., 2013, 31, (3), pp. 255–274 (doi: 10.1016/j.imavis.2013.01.003).
4. 4)
  - 8. Merino, C., Mirmehdi, M.: ‘A framework towards realtime detection and tracking of text’. Camera Based Document Analysis and Recognition, 2007, pp. 10–17.
5. 5)
  - 23. Wang, K., Babenko, B., Belongie, S.: ‘End-to-end scene text recognition’. Int. Conf. on Computer Vision, 2011, pp. 1457–1464.
6. 6)
  - 14. Tanaka, M., Goto, H.: ‘Autonomous text capturing robot using improved DCT feature and text tracking’. Int. Conf. on Document Analysis and Recognition, 2007, 2, pp. 1178–1182.
7. 7)
  - 19. Pan, Y.F., Hou, X., Liu, C.L.: ‘A hybrid approach to detect and localize texts in natural scene images’, IEEE Trans. Image Process., 2011, 20, (3), pp. 800–813 (doi: 10.1109/TIP.2010.2070803).
8. 8)
  - 33. Klein, G., Murray, D.: ‘Parallel tracking and mapping for small AR workspaces’. ISMAR, 2007, pp. 225–234.
9. 9)
  - 17. Na, Y., Wen, D.: ‘An effective video text tracking algorithm based on sift feature and geometric constraint’. Advances in Multimedia Information Processing, 2010, pp. 392–403.
10. 10)
  - R. Lienhart , A. Wernicke . Localizing and segmenting text in images and videos. IEEE Trans. Circuits Syst. Video Technol. , 4 , 256 - 267
11. 11)
  - 24. Mishra, A., Alahari, K., Jawahar, C.: ‘Top-down and bottom-up cues for scene text recognition’. Computer Vision and Pattern Recognition, 2012, pp. 2687–2694.
12. 12)
  - 28. de Berg, M., van Kreveld, M., Overmars, M., Schwarzkopf, O.: ‘Computational geometry: algorithms and applications’ (Springer-Verlag, 2000, 2nd edn.).
13. 13)
  - 4. Chen, H., Tsai, S., Schroth, G., Chen, D., Grzeszczuk, R., Girod, B.: ‘Robust text detection in natural images with edge-enhanced maximally stable external regions’. Int. Conf. on Image Processing, 2011, pp. 2609–2612.
14. 14)
  - 30. Toussaint, G.: ‘Solving geometric problems with the rotating calipers’. Mediterranean Electrotechnical Conf., 1983, pp. 10–17.
15. 15)
  - 5. Lucas, S.M., Panaretos, A., Sosa, L., Tang, A., Wong, S., Young, R.: ‘ICDAR 2003 robust reading competitions’. Int. Conf. on Document Analysis and Recognition, 2003, pp. 682–687.
16. 16)
  - 35. Bernardin, K., Stiefelhagen, R.: ‘Evaluating multiple object tracking performance: the CLEAR MOT metrics’, J. Image Video Process., 2008, 2008, pp. 1:1–1:10.
17. 17)
  - 12. Myers, G.K., Burns, B.: ‘A robust method for tracking scene text in video imagery’. Camera Based Document Analysis and Recognition, 2005, vol. 1.
18. 18)
  - 18. Minetto, R., Thome, N., Cord, M., Leite, N.J., Stolfi, J.: ‘Snoopertrack: text detection and tracking for outdoor videos’. Int. Conf. on Image Processing, 2011, pp. 505–508.
19. 19)
  - 31. Doucet, A., de Freitas, J., Gordon, N.: ‘Sequential Monte Carlo methods in practice’ (Springer-Verlag, 2001).
20. 20)
  - 15. Tanaka, M., Goto, H.: ‘Text-tracking wearable camera system for visually-impaired people’. Int. Conf. on Pattern Recognition, 2008, pp. 1–4.
21. 21)
  - 6. Myers, G.K., Bolles, R.C., Luong, Q.-T., Herson, J.A., Aradhye, H.B.: ‘Rectification and recognition of text in 3-D scenes’, Int. J. Doc. Anal. Recognit., 2005, 7, pp. 147–158 (doi: 10.1007/s10032-004-0133-4).
22. 22)
  - 26. Merino-Gracia, C., Lenc, K., Mirmehdi, M.: ‘A head-mounted device for recognizing text in natural scenes’, in Iwamura, M., Shafait, F.(Eds.): ‘Camera based document analysis and recognition’, 2012, (LNCS, 7139), pp. 29–41.
23. 23)
  - 3. Neumann, L., Matas, J.: ‘Real-time scene text localization and recognition’. Computer Vision and Pattern Recognition, 2012, pp. 3538–3545.
24. 24)
  - 32. Wan, E., Van Der Merwe, R.: ‘The unscented Kalman filter for nonlinear estimation’. Adaptive Systems for Signal Processing, Communications, and Control, 2000, pp. 153–158.
25. 25)
  - 11. Gllavata, J., Ewerth, R., Freisleben, B.: ‘Tracking text in MPEG videos’. Int. Conf. on Multimedia, 2004, pp. 240–243.
26. 26)
  - 13. Shiratori, H., Goto, H., Kobayashi, H.: ‘An efficient text capture method for moving robots using DCT feature and text tracking’. Int. Conf. on Pattern Recognition, 2006, pp. 1050–1053.
27. 27)
  - 21. Phan, T.Q., Shivakumara, P., Lu, T., Tan, C.L.: ‘Recognition of video text through temporal integration’. Int. Conf. on Document Analysis and Recognition, 2013, pp. 589–593.
28. 28)
  - 20. Hartley, R.I., Zisserman, A.: ‘Multiple view geometry in computer vision’ (Cambridge University Press, 2004, 2nd edn.).
29. 29)
  - D.G. Lowe . Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis , 2 , 91 - 110
30. 30)
  - 27. Targhi, A., Hayman, E., Eklundh, J., Shahshahani, M.: ‘The Eigen-transform & applications’. Asian Conf. of Computer Vision, I, 2006, pp. 70–79.
31. 31)
  - 1. Epshtein, B., Ofek, E., Wexler, Y.: ‘Detecting text in natural scenes with stroke width transform’. Computer Vision and Pattern Recognition, 2010, pp. 2963–2970.
32. 32)
  - H. Li , D. Doermann , O. Kia . Automatic text detection and tracking in digital video. IEEE Trans. Image Process. , 1 , 147 - 156
33. 33)
  - 16. Goto, H., Tanaka, M.: ‘Text-tracking wearable camera system for the blind’. Int. Conf. on Document Analysis and Recognition, 2009, pp. 141–145.
34. 34)
  - 7. Merino-Gracia, C., Mirmehdi, M., Sigut, J., González-Mora, J.L.: ‘Fast perspective recovery of text in natural scenes’, Image Vis. Comput., 2013, 31, pp. 714–724 (doi: 10.1016/j.imavis.2013.07.002).
35. 35)
  - 22. Mosleh, A., Bouguila, N., Hamza, A.: ‘Automatic inpainting scheme for video text detection and removal’, Trans. Image Process., 2013, 22, (11), pp. 4460–4472 (doi: 10.1109/TIP.2013.2273672).

Login

Not registered yet?

Share

Tools

Login to add to favourites

Key

Real-time text tracking in natural scenes

References

Related content