access icon free Dynamic error spectrum for estimation performance evaluation: a case study on interacting multiple model algorithm

© The Institution of Engineering and Technology
Received 03/04/2013, Accepted 26/07/2013, Revised 28/06/2013, Published 18/09/2013

The commonly used root-mean-square error for estimation performance evaluation is easily dominated by large error terms. So many new alternative absolute metrics have been provided in X. R. Li's work. However, each of these metrics only reflects one narrow aspect of estimation performance, respectively. A comprehensive measure, error spectrum, was presented aggregating all these incomprehensive measures. However, when being applied to dynamic systems, this measure will have three dimensions over the total time span, which is not intuitive and difficult to be analysed. To overcome its drawbacks, a new metric, dynamic error spectrum (DES), is proposed in this study to extend the error spectrum measure to dynamic systems. Three forms under different application backgrounds are given, one of which is balanced taking into account both good and bad behaviour of an estimator and so can provide more impartial evaluation results. It can be applied to a variety of dynamic systems directly. Then the challenge in performance evaluation of the interacting multiple model (IMM) algorithm is considered, and the IMM algorithm is chosen as the testing case to illustrate the superiority of the DES metric. The simulation results validate its utility and effectiveness.

Inspec keywords: estimation theory; mean square error methods

Other keywords: multiple model algorithm; DES metric; dynamic systems; dynamic error spectrum; root-mean-square error; IMM algorithm; estimation performance; performance evaluation; interacting multiple model algorithm

Subjects: Statistics; Numerical analysis; Interpolation and function approximation (numerical analysis); Interpolation and function approximation (numerical analysis); Probability theory, stochastic processes, and statistics; Other topics in statistics; Numerical approximation and analysis; Other topics in statistics

References

    1. 1)
    2. 2)
    3. 3)
    4. 4)
    5. 5)
    6. 6)
    7. 7)
    8. 8)
    9. 9)
    10. 10)
    11. 11)
    12. 12)
    13. 13)
    14. 14)
      • 17. Nadarajah, N., Tharmarasa, R., McDonald, M., Kirubarajan, T.: ‘IMM forward filtering and backward smoothing for maneuvering target tracking’, IEEE Trans. Aerosp. Electron. Syst., 2012, 48, (3), pp. 2673–2678 (doi: 10.1109/TAES.2012.6237617).
    15. 15)
      • 1. Li, X.R., Zhao, Z.L.: ‘Measures of performance for evaluation of estimators and filters’. Proc. 2001 SPIE Conf. Signal and Data Processing of Small Targets, November 2001, pp. 530–541.
    16. 16)
      • 16. Johnston, L.A., Krishnamurthy, V.: ‘An improvement to the interacting multiple model (IMM) algorithm’, IEEE Trans. Signal Process., 2001, 49, (12), pp. 2909–2923 (doi: 10.1109/78.969500).
    17. 17)
      • 18. Benoudnine, H., Keche, M., Ouamri, A., Woolfson, M.: ‘New efficient schemes for adaptive selection of the update time in the IMMJPDAF’, IEEE Trans. Aerosp. Electron. Syst., 2012, 48, (1), pp. 197–214 (doi: 10.1109/TAES.2012.6129630).
    18. 18)
      • 22. Seah, C.E., Hwang, I.: ‘Performance analysis of Kalman filter based hybrid estimation algorithms’. Proc. 17th IFAC World Congress, Seoul, Korea, July 2008, pp. 13498–13503.
    19. 19)
      • 14. Bar-Shalom, Y., Challa, S., Blom, H.A.P.: ‘IMM estimator versus optimal estimator for hybrid systems’, IEEE Trans. Aerosp. Electron. Syst., 2005, 41, (3), pp. 986–991 (doi: 10.1109/TAES.2005.1541443).
    20. 20)
      • 8. Zhang, Z., Ji, L., Huang, Z., Wu, J.: ‘Multi-model adaptation for thigh movement estimation using accelerometers’, IET Signal Process., 2011, 5, (8), pp. 709–716 (doi: 10.1049/iet-spr.2010.0146).
    21. 21)
      • 12. Blom, H.A.P.: ‘An efficient filter for abruptly changing systems’. Proc. 23rd IEEE Conf. Decision and Control, Las, Vegas, NV, December 1984, pp. 656–658.
    22. 22)
      • 7. Maybeck, P.S.: ‘Stochastic models, estimation and control’ (Academic Press, New York, 1982).
    23. 23)
      • 20. Drummond, O.E., Li, X.R., He, C.: ‘Comparison of various static multiple-model estimation algorithms’. Proc. 1998 SPIE Conf. Signal and Data Processing of Small Targets, April 1998, vol. 3373, pp. 510–527.
    24. 24)
      • 10. Li, W., Jia, Y.: ‘State estimation for jump Markov linear systems by variational Bayesian approximation’, IET Control Theory Appl., 2012, 6, (2), pp. 319–326 (doi: 10.1049/iet-cta.2011.0167).
    25. 25)
      • 11. Lan, J., Li, X.R., Jilkov, V.P., Mu, C.: ‘Second-order Markov chain based multiple-model algorithm for maneuvering target tracking’, IEEE Trans. Aerosp. Electron. Syst., 2013, 49, (1), pp. 3–19 (doi: 10.1109/TAES.2013.6404088).
    26. 26)
      • 19. Seah, C., Hwang, I.: ‘Algorithm for performance analysis of the IMM algorithm’, IEEE Trans. Aerosp. Electron. Syst., 2011, 47, (2), pp. 1114–1124 (doi: 10.1109/TAES.2011.5751246).
    27. 27)
      • 4. Bar-Shalom, Y., Li, X.R., Kirubarajan, T.: ‘Estimation with applications to tracking and navigation: theory, algorithms, and software’ (Wiley, New York, 2001).
    28. 28)
      • 5. Bar-Shalom, Y.: ‘Multitarget-multisensor tracking: advanced applications’ (Artech House, Norwood, MA, 1992).
    29. 29)
      • 13. Blom, H.A.P., Bar-Shalom, Y.: ‘The interacting multiple model algorithm for systems with Markovian switching coefficients’, IEEE Trans. Autom. Control, 1988, 33, (8), pp. 780–783 (doi: 10.1109/9.1299).
    30. 30)
      • 15. Guu, J.A., Wei, C.H.: ‘Maneuvering target tracking using IMM method at high measurement frequency’, IEEE Trans. Aerosp. Electron. Syst., 1991, 27, (3), pp. 514–519 (doi: 10.1109/7.81433).
    31. 31)
      • 3. Li, X.R., Zhao, Z.L., Duan, Z.S.: ‘Error spectrum and desirability level for estimation performance evaluation’. Proc. Workshop on Estimation, Tracking and Fusion: A Tribute to Fred Daum, Monterey, CA, USA, May 2007.
    32. 32)
      • 2. Li, X.R., Zhao, Z.L.: ‘Evaluation of estimation algorithms. Part I: incomprehensive measures of performance’, IEEE Trans. Aerosp. Electron. Syst., 2006, 42, (4), pp. 1340–1358 (doi: 10.1109/TAES.2006.314576).
    33. 33)
      • 9. Sigalov, D., Michaeli, T., Oshman, Y.: ‘A unified approach to state estimation problems under data and model uncertainties’. Proc. 15th Int. Conf. Information Fusion, 2012, pp. 2569–2576.
    34. 34)
      • 6. Li, X.R., Jilkov, V.P.: ‘Survey of maneuvering target tracking. Part V: multiple-model methods’, IEEE Trans. Aerosp. Electron. Syst., 2005, 41, (4), pp. 1255–1321 (doi: 10.1109/TAES.2005.1561886).
    35. 35)
      • 23. Osborne, R.W., Blair, W.D.: ‘Update to the hybrid conditional averaging performance prediction of the IMM algorithm’, IEEE Trans. Aerosp. Electron. Syst., 2011, 47, (4), pp. 2967–2974 (doi: 10.1109/TAES.2011.6034677).
    36. 36)
      • 21. Busch, M.T., Blackman, S.S.: ‘Evaluation of IMM filtering for an air defence system application’. Proc. 1995 SPIE Conf. Signal and Data Processing of Small Targets, September 1995, vol. 2561, pp. 435–447.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-spr.2013.0134
Loading

Related content

content/journals/10.1049/iet-spr.2013.0134
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading