Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free Model-based margin estimation for hidden Markov model learning and generalisation

Recently, speech scientists have been motivated by the great, success of building margin-based classifiers, and have thus proposed novel methods to estimate continuous-density hidden Markov model (HMM) for automatic speech recognition (ASR) according to the notion that the decision boundaries determined by the estimated HMMs attain the maximum classification margin as in learning support vector machines. Although a good performance has been observed, the margin used in the ASR community is often specified as a parameter that has no explicit relationship with the HMM parameters. The issues of how the margin is related to the HMM parameters and how it directly characterises the generalisation capability of HMM-based classifiers have not been addressed so far in the community. In this study, the authors attempt to formulate the margin used in the soft margin estimation framework as a function of the HMM parameters. The key idea is to relate the standard distance-based margin with the concept of divergence among competing HMM state Gaussian mixture model densities. Experimental results show that the proposed model-based margin function is a good indication about the quality of HMMs on a given ASR task without the conventional needs of running experiments extensively using a separate set of test samples.

References

    1. 1)
      • 9. Katagiri, S., Juang, B.H., Lee, C.H.: ‘Pattern recognition using a family of design algorithms based upon generalized probability descent method’, Proc. IEEE, 1998, 86, (11), pp. 23452373 (doi: 10.1109/5.726793).
    2. 2)
      • 5. Sha, F., Saul, L.K.: ‘Large margin hidden Markov models for automatic speech recognition’, in: Schlkopf, B., Platt, J.C., Hofmann, T., Eds., ‘Advances in neural information processing systems 19’ (MIT Press, 2007), pp. 12491256.
    3. 3)
      • 15. Li, J., Lee, C.H.: ‘On a generalization of margin-based discriminative training to robust speech recognition’. INTERSPEECH, 2008, pp. 19921995.
    4. 4)
      • 16. Xiao, X., Li, J., Chng, E., Li, H., Lee, C.H.: ‘A study on the generalization capability of acoustic models for robust speech recognition’, IEEE Trans. Audio, Speech, Lang. Process., 2010, 18, pp. 11581169 (doi: 10.1109/TASL.2009.2031236).
    5. 5)
      • 1. Burges, C.: ‘A tutorial on support vector machines for pattern recognition’, Data Min. Knowl. Discov., 1998, 2, (2), pp. 121167 (doi: 10.1023/A:1009715923555).
    6. 6)
      • 13. Kullback, S.: ‘Information theory and statistics’ (Dover, New York, 1968).
    7. 7)
      • 6. Li, J., Siniscalchi, S.M., Lee, C.H.: ‘Approximate test risk minimization through soft margin estimation’. Proc. ICASSP, Honolulu, Hawaii, USA, 2007, pp. IV-653IV-656.
    8. 8)
      • 17. Leonard, R.G.: ‘A database for speaker-independent digit recognition’. Proc. IEEE ICASSP, San Diego, CA, USA, 1984, pp. 328331.
    9. 9)
      • 14. Du, J., Liu, P., Zhou, F.K.S., Wang, J.L., R.H.: ‘Minimum divergence based discriminative training’. Proc. ICASSP, Pittsburgh, PA, USA, 2006, pp. 24102413.
    10. 10)
      • 7. Li, J., Yuan, M., Lee, C.H.: ‘Approximate test risk bound minimization through soft margin estimation’, IEEE Trans. Audio Speech, Lang. Process., 2007b, 15, (8), pp. 23932404 (doi: 10.1109/TASL.2007.906178).
    11. 11)
      • 3. Yu, D., Deng, L., He, X., Acero, A.: ‘Use of incrementally regulated discriminative margins in MCE training for speech recognition’. Proc. Interspeech, Pittsburgh, PA, USA, 2006, pp. 24182421.
    12. 12)
      • 11. Fukunaga, K.: ‘Introduction to statistical pattern recognition’ (Academic Press Inc., New York, 1990, 2nd edn.).
    13. 13)
      • 2. Jiang, H., Li, X., Liu, C.: ‘Large margin hidden Markov models for speech recognition’, IEEE Trans. Audio, Speech Lang. Process., 2006, 14, (5), pp. 15841595 (doi: 10.1109/TASL.2006.879805).
    14. 14)
      • 18. Pearce, D., Hirsch, H.G.: ‘The aurora experimental framework for the performance evaluation of speech recognition system under noisy condition’. Proc. ICSLP, Beijing, China, 2000, pp. 2932.
    15. 15)
      • 19. Hershey, J., Olsen, P.: ‘Approximating the Kullback Leibler divergence between Gaussian mixture models’. Proc. ICASSP, Honolulu, Hawaii, USA, 2007, pp. IV 317IV 320.
    16. 16)
      • 12. Huang, C.S., Wang, H.C., Lee, C.H.: ‘A study on model-based error rate estimation for automatic speech recognition’, IEEE Trans. Speech Audio Process., 2003, 11, (6), pp. 581589 (doi: 10.1109/TSA.2003.818030).
    17. 17)
      • 8. Bahl, L.R., Brown, P.F., deSouza, P.V., Mercer, R.L.: ‘Maximum mutual information estimation of HMM parameters for speech recognition’. Proc. ICASSP, Tokyo, Japan, 1986.
    18. 18)
      • 4. Li, J., Yuan, M., Lee, C.H.: ‘Soft margin estimation of hidden Markov model parameters’. Proc. Interspeech, Pittsurgh, USA, 2006, pp. 24222425.
    19. 19)
      • 10. Juang, B.H., Chou, W., Lee, C.H.: ‘Minimum classification minimum error rate methods for speech recognition’, IEEE Trans. Speech Audio Process., 1997, 5, pp. 257265 (doi: 10.1109/89.568732).
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-spr.2013.0036
Loading

Related content

content/journals/10.1049/iet-spr.2013.0036
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address