access icon free Weighted ensemble based on 0-1 matrix decomposition

© The Institution of Engineering and Technology
Received 13/10/2012, Published

A simple effective ensemble method is proposed in which individual classifiers are combined with the weight coefficients obtained by decomposition for the 0-1 matrix. The 0-1 matrix is introduced to denote individual classifiers of the ensemble and is constructed based on the prediction labels of individuals and the true labels. The weight coefficients of individuals are obtained by singular value decomposition for the 0-1 matrix based on linear mapping. In particular, the square of elements of the right singular vector corresponding to the maximum singular value are as the weight coefficients of individuals, and it is proved theoretically that it minimises the upper bound of ensemble error. Experimental results illustrate that the proposed method improves the performance of classification compared against standard ensemble strategies.

Inspec keywords: singular value decomposition; pattern classification

Other keywords: singular value decomposition; prediction labels; individual classifiers; weight coefficients; linear mapping; upper bound minimisation; 0-1 matrix decomposition; effective ensemble method; ensemble error; true labels; weight ensemble

Subjects: Pattern recognition; Algebra

References

    1. 1)
      • 1. Kuncheva, , L.I.: ‘Combining pattern classifiers: Methods and algorithms’ (Wiley Interscience, New York, 2005).
    2. 2)
      • 8. Breiman, , L.: ‘Bagging predictors’, Mach. Learn., 1995, 24, pp. 123–140.
    3. 3)
      • 6. Margineantu, , D.D., Dietterich, , T.G.: ‘Pruning adaptive boosting’. Proc. of 14th Int. Conf. on Machine Learning, 1997, New York, USA, pp. 211–218..
    4. 4)
      • 2. Su, , Y., Shan, , S., Chen, , X., Gao, , W.: ‘Hierarchical ensemble of global and local classifiers for face recognition’, IEEE Trans. Image Process., 2009, 18, pp. 1885–1896 (doi: 10.1109/TIP.2009.2021737).
    5. 5)
      • 7. Kalman, , D.: ‘A singularly valuable decomposition: the SVD of a matrix’, College Math J., 1996, 27, pp. 2–23 (doi: 10.2307/2687269).
    6. 6)
      • 3. Tao, , D., Tang, , X., et al.: ‘Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, pp. 1088–1099 (doi: 10.1109/TPAMI.2006.134).
    7. 7)
      • 4. Melville, , P., Mooney, , R.J.: ‘Creating diversity in ensembles using artificial data’, Inf. Fusion, 2004, 6, pp. 99–111 (doi: 10.1016/j.inffus.2004.04.001).
    8. 8)
      • 5. Sylvester, , J., Chawla, , N.V.: ‘Evolutionary ensemble creation and thinning’. Int. Joint Conf. on Neural Networks, Brisbane, Australia, 2006, pp. 5148–5155.
    9. 9)
    10. 10)
    11. 11)
    12. 12)
      • Margineantu, D.D., Dietterich, T.G.: `Pruning adaptive boosting', Proc. of 14th Int. Conf. on Machine Learning, 1997, New York, USA, p. 211–218.
    13. 13)
    14. 14)
      • Sylvester, J., Chawla, N.V.: `Evolutionary ensemble creation and thinning', Int. Joint Conf. on Neural Networks, 2006, Brisbane, Australia, p. 5148–5155.
    15. 15)
      • L.I. Kuncheva . (2005) Combining pattern classifiers: Methods and algorithms.
    16. 16)
http://iet.metastore.ingenta.com/content/journals/10.1049/el.2012.3528
Loading

Related content

content/journals/10.1049/el.2012.3528
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading