© The Institution of Engineering and Technology
Skin segmentation and tracking play an important role in sign language recognition. A framework for segmenting and tracking skin objects from signing videos is described. It mainly consists of two parts: a skin colour model and a skin object tracking system. The skin colour model is first built based on the combination of support vector machine active learning and region segmentation. Then, the obtained skin colour model is integrated with the motion and position information to perform segmentation and tracking. The tracking system is able to predict occlusions among any of the skin objects using a Kalman filter (KF). Moreover, the skin colour model can be updated with the help of tracking to handle illumination variation. Experimental evaluations using real-world gesture videos and comparison with other existing algorithms demonstrate the effectiveness of the proposed work.
References
-
-
1)
-
M. Soriano ,
B. Martinkauppi ,
S. Huovinen ,
M. Laaksonen
.
Adaptive skin color modeling using the skin locus for selecting training pixels.
Pattern Recognit.
,
681 -
690
-
2)
-
S.L. Phung ,
A. Bouzerdoum ,
D. Chai
.
Skin segmentation using colour pixel classification: analysis and comparison.
IEEE Trans. Pattern Anal. Mach. Intell.
,
148 -
154
-
3)
-
V. Vapnik
.
(1995)
The nature of statistical learning theory.
-
4)
-
M. Isard ,
A. Blake
.
Condensation-conditional density propagation for visual tracking.
Int. J. Comput. Vis.
,
1 ,
5 -
28
-
5)
-
T. Wu ,
C. Lin ,
R. Weng
.
Probability estimates for multi-class classification by pairwise coupling.
J. Mach. Learn. Res.
,
975 -
1005
-
6)
-
C.K. Chui ,
G. Chen
.
(1998)
Kalman filtering with real-time applications.
-
7)
-
Mammen, J., Chaudhuri, S., Agrawal, T.: `Simultaneous tracking of both hands by estimation of erroneous observations', Proc. British Machine Vision Conf., September 2001, Manchester, UK, p. 83–92.
-
8)
-
Zhu, Q., Wu, C.T., Cheng, K.T., Wu, Y.L.: `An adaptive skin model and its application to objectionable image filtering', Proc. ACM Multimedia, October 2004, New York, USA, p. 56–63.
-
9)
-
Rehg, J., Kanade, T.: `Visual tracking of high DOF articulated structures: an application to human hand tracking', Proc. 3rd European Conf. Computer Vision, May 1994, Stockholm, Sweden, p. 35–46.
-
10)
-
Wu, Y., Huang, T.S.: `Colour tracking by transductive learning', Proc. IEEE Conf. Computer Vision Pattern Recognition, June 2000, SC, USA, p. 33–138.
-
11)
-
C. Shan ,
T. Tan ,
Y. Wei
.
Real-time hand tracking using a mean shift embedded particle filter.
Pattern Recognit.
,
1958 -
1972
-
12)
-
Sherrah, J., Gong, S.: `Tracking discontinuous motion using Bayesian inference', Proc. European Conf. Computer Vision, June 2000, Dublin, Ireland, p. 150–166.
-
13)
-
A. Shamaie ,
A. Sutherland
.
Hand tracking in bimanual movements.
Image Vision Comput.
,
1131 -
1149
-
14)
-
S.J. McKenna ,
S. Gong ,
Y. Raja
.
Tracking colour objects using adaptive mixture models.
Image Vis. Comput.
,
225 -
231
-
15)
-
Y. Deng ,
B.S. Manjunath
.
Unsupervised segmentation of colortexture regions in images and video.
IEEE Trans. Pattern Anal. Mach. Intell.
,
8 ,
800 -
810
-
16)
-
Stenger, B., Mendonca, P.R.S., Cipolla, R.: `Model-based 3D tracking of an articulated hand', Proc. Conf. Computer Vision and Pattern Recognition, December 2001, Hawaii, USA, p. 310–315.
-
17)
-
L. Sigal ,
S. Sclaroff ,
V. Athitsos
.
Skin colour-based video segmentation under time-varying illumination.
IEEE Trans. Pattern Anal. Mach. Intell.
,
7 ,
862 -
877
-
18)
-
M. Storring ,
H. Andersen ,
E. Granum
.
Physics-based modelling of human skin colour under mixed illuminants.
Robot. Auton. Syst.
,
131 -
142
-
19)
-
Lu, S., Metaxas, D., Samaras, D., Oliensis, J.: `Using multiple cues for hand tracking and model refinement', Proc. Conf. Computer Vision and Pattern Recognition, June 2003, Wisconsin, USA, p. 443–450.
-
20)
-
Coogan, T., Awad, G., Han, J., Sutherland, A.: `Real time hand gesture recognition including hand segmentation and tracking', Proc. Int. Symp. Visual Computing, November 2006, Nevada, USA, p. 495–504°.
-
21)
-
W. Gao ,
J. Ma ,
J. Wu ,
C. Wang
.
Large vocabulary sign language recognition based on HMM/ANN/DP.
Int. J. Pattern Recognit. Art. Intell.
,
5 ,
587 -
602
-
22)
-
M. Jones ,
J. Rehg
.
Statistical color models with application to skin detection.
Int. J. Comput. Vis.
,
1 ,
81 -
96
-
23)
-
C. Wren ,
A. Azarbayejani ,
T. Darrell ,
A. Pentland
.
Pfinder: real-time tracking of the human body.
IEEE Trans. Pattern Anal. Mach. Intell.
,
780 -
785
-
24)
-
Tong, S., Chang, E.: `Support vector machine active learning for image retrieval', Proc. ACM Multimedia, September 2001, Ottawa, Canada, p. 107–118.
-
25)
-
Martin, J., Devin, V., Crowley, J.: `Active hand tracking', Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, April 1998, Nara, Japan, p. 573–578.
-
26)
-
S. Ong ,
S. Ranganath
.
Automatic sign language analysis: a survey and the future beyond lexical meaning.
IEEE Trans. Pattern Anal. Mach. Intell.
,
6 ,
873 -
891
-
27)
-
T. Starner ,
J. Weaver ,
A. Pentland
.
Real-time American sign language recognition using desk and wearable computer based video.
IEEE Trans. Pattern Anal. Mach. Intell.
,
12 ,
1371 -
1375
-
28)
-
Schohn, G., Cohn, D.: `Less is more: active learning with support vector machines', Proc. Int. Conf. Machine Learning, June 2000, Stanford, USA, p. 839–846.
-
29)
-
M.-H. Yang ,
N. Ahuja ,
M. Tabb
.
Extraction of 2D trajectories and its application to hand gesture recognition.
IEEE Trans. Pattern Anal. Mach. Intell.
,
8 ,
1061 -
1074
-
30)
-
G. McAllister ,
S.J. McKenna ,
I.W. Picketts
.
Hand tracking for behaviour understanding.
Image Vision Comput.
,
827 -
840
-
31)
-
D. Chai ,
K.N. Ngan
.
Face segmentation using skin-colour map in videophone applications.
IEEE Trans. Circuits Syst. Video Technol.
,
551 -
564
-
32)
-
Awad, G., Han, J., Sutherland, A.: `A unified system for segmentation and tracking of face and hands in sign language recognition', Proc. Int. Conf. Pattern Recognition, August 2006, Hong Kong, China, p. 239–242.
-
33)
-
Imagawa, K., Lu, S., Igi, S.: `Colour-based hands tracking system for sign language recognition', Proc. IEEE Int. Conf. Automatic Face and Gesture Recognition, April 1998, Nara, Japan, p. 462–467.
-
34)
-
A. Nayak ,
S. Chaudhuri
.
Automatic illumination correction for scene enhancement and object tracking.
Image Vision Comput.
,
949 -
959
-
35)
-
F.-S. Chen ,
C.-M. Fu ,
C.-L. Huang
.
Hand gesture recognition using a real-time tracking method and hidden Markov Models.
Image Vision Comput.
,
745 -
758
-
36)
-
Kovac, J., Peer, P., Solina, F.: `Human skin colour clustering for face detection', Proc. EUROCON 2003, September 2003, Ljubljana, Slovenia, p. 144–148.
-
37)
-
Vezhnevets, V., Sazonov, V., Andreeva, A.: `A survey on pixel-based skin color detection techniques', Proc. Graphicon, September 2003, Moscow, Russia, p. 85–92.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi_20080006
Related content
content/journals/10.1049/iet-cvi_20080006
pub_keyword,iet_inspecKeyword,pub_concept
6
6