access icon openaccess Locality constrained dictionary learning for non-linear dimensionality reduction and classification

This is an open access article published by the IET under the Creative Commons Attribution-NonCommercial-NoDerivs License (http://creativecommons.org/licenses/by-nc-nd/3.0/)
Received 20/12/2015, Accepted 15/08/2016, Revised 26/07/2016, Published 23/08/2016

In view of the incremental dimensionality reduction problem of existing non-linear dimensionality reduction methods, a novel algorithm, based on locality constrained dictionary learning (LCDL), is proposed in this study. During the dictionary learning process, the neighbourhood size of some potential landmarks on a non-linear manifold is constrained to maintain the intrinsic local geometric feature of the datasets. Meanwhile, to improve the dictionary's discrimination ability, a structured dictionary is learnt by LCDL, whose sub-dictionaries are class-specific. Then sparse coding and its reconstruction errors are used for classification. The experimental results of dimensionality reduction prove that, compared with the existing methods, the proposed method can solve the out of sample extension and large-scale datasets problems efficiently. In addition, the experimental results of face, gender, and object category classification demonstrate that the authors’ algorithm outperforms some competing dictionary learning methods.

Inspec keywords: learning (artificial intelligence); pattern classification

Other keywords: nonlinear dimensionality reduction method; object category classification; nonlinear dimensionality classification; reconstruction errors; locality constrained dictionary learning process; incremental dimensionality reduction problem; sparse coding; LCDL; intrinsic local geometric feature; nonlinear manifold; dictionary discrimination ability

Subjects: Data handling techniques; Knowledge engineering techniques

References

    1. 1)
      • 17. Wang, J., Yang, J., Yu, K., et al: ‘Locality constrained linear coding for image classification’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, San Francisco, USA, June 2010, pp. 3360–3367.
    2. 2)
      • 3. Ye, Q.W., Sun, Y., Wang, X.D., et al: ‘An improved LLE algorithm with sparse constraint’, J. Comput. Theor. Nanos., 2013, 10, (12), pp. 2872–2876.
    3. 3)
      • 10. Ramirez, I., Sprechmann, P., Sapiro, G.: ‘Classification and clustering via dictionary learning with structured incoherence and shared features’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, San Francisco, USA, June 2010, pp. 3501–3508.
    4. 4)
      • 20. Tropp, J., Gilbert, A.: ‘Signal recovery from random measurements via orthogonal matching pursuit’, IEEE Trans. Inf. Theory, 2007, 53, (12), pp. 4655–4666.
    5. 5)
      • 18. Zhou, Y., Liu, K., Barner, K.E.: ‘Non-rigid 3D shape recognition via dictionary learning’. Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, Vancouver, BC, May 2013, pp. 3502–3506.
    6. 6)
      • 25. Griffin, G., Holub, A., Perona, P.: ‘Caltech-256 object category data set’ (California Institute of Technology, 2007), pp. 1–20.
    7. 7)
      • 24. Li, F.F., Fergus, R., Perona, P.: ‘Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, Washington, USA, June 2004, pp. 178–187.
    8. 8)
      • 27. Krizhevsky, A.: ‘Learning multiple layers of features from tiny images’, 2009, pp. 1–60.
    9. 9)
      • 12. Xie, C.H., Tan, J.Q., Chen, P., et al: ‘Multiple instance learning tracking method with local sparse representation’, IET Comput. Vis., 2013, 7, (5), pp. 320–334.
    10. 10)
      • 9. Cheng, G., Zhou, P.C., Han, J.W., et al: ‘Auto-encoder-based shared mid-level visual dictionary learning for scene classification using very high resolution remote sensing images’, IET Comput. Vis., 2015, 9, (5), pp. 639–647.
    11. 11)
      • 21. Boothby, W.M.: ‘An introduction to differentiable manifolds and riemannian geometry’ (Academic Press, 2003, 2nd edn.).
    12. 12)
      • 2. Wang, R.P., Shan, S.G., Chen, X.L., et al: ‘Maximal linear embedding for dimensionality reduction’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (9), pp. 1776–1792.
    13. 13)
      • 14. Zhou, Y., Barner, K.E.: ‘Locality constrained dictionary learning for nonlinear dimensionality reduction’, IEEE Signal Process. Lett., 2013, 20, (4), pp. 335–338.
    14. 14)
      • 16. Schnass, K., Vandergheynst, P.: ‘Dictionary learning based dimensionality reduction for classification’. Proc. IEEE Int. Conf. Signal Processing Society, Julians, Malta, March 2008, pp. 780–785.
    15. 15)
      • 1. Maaten, L.V.D., Postma, E., Herik, J.V.D.: ‘Dimensionality reduction: a comparative review’, J. Mach. Learn. Res., 2009, 10, (1), pp. 1–22.
    16. 16)
      • 23. Jiang, Z., Lin, Z., Davis, L.: ‘Label consistent k-svd: learning a discriminative dictionary for recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2013, 35, (11), pp. 2651–2664.
    17. 17)
      • 28. Torralba, A., Fergus, R., Freeman, W.: ‘80 million tiny images: a large data set for nonparametric object and scene recognition’, IEEE Trans. Pattern Anal. Mach. Intell., 2008, 30, (11), pp. 1958–1970.
    18. 18)
      • 30. Bo, L., Ren, X., Fox, D.: ‘Kernel descriptors for visual recognition’. Proc. Int. Conf. Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 2010, pp. 244–252.
    19. 19)
      • 13. Yang, J., Yu, K., Gong, Y., et al: ‘Linear spatial pyramid matching using sparse coding for image classification’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, Miami, FL, USA, June 2009, pp. 1794–1801.
    20. 20)
      • 8. Leng, Y.Q., Zhang, L., Yang, J.W.: ‘Locally linear embedding algorithm based on OMP for incremental learning’. Proc. Int. Joint Conf. Neural Networks, Beijing, China, July 2014, pp. 3100–3107.
    21. 21)
      • 11. Yang, M., Zhang, L., Feng, X.C., et al: ‘Fisher discrimination dictionary learning for sparse representation’. Proc. Int. Conf. Computer Vision, Barcelona, Spain, November 2011, pp. 543–550.
    22. 22)
      • 19. Aharon, M., Elad, M., Bruckstein, A.: ‘K-svd: An algorithm for designing overcomplete dictionaries for sparse representation’, IEEE Trans. Signal Process., 2006, 54, (11), pp. 4311–4322.
    23. 23)
      • 7. Kapoor, R., Gupta, R.: ‘Morphological mapping for non-linear dimensionality reduction’, IET Comput. Vis., 2015, 9, (2), pp. 226–233.
    24. 24)
      • 26. Lazebnik, S., Schmid, C., Ponce, J.: ‘Beyond bags of features: spatial pyramid matching for recognizing natural scene categories’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, New York, USA, June 2006, pp. 2169–2178.
    25. 25)
      • 22. Wright, J., Yang, A.Y., Ganesh, A., et al: ‘Robust face recognition via sparse representation’, IEEE Trans. Pattern Anal. Mach. Intell., 2009, 31, (2), pp. 210–227.
    26. 26)
      • 15. Zhao, Z., Feng, G.C.: ‘A dictionary-based algorithm for dimensionality reduction and data reconstruction’. Proc. IEEE Int. Conf. Pattern Recognition, Stockholm, Sweden, August 2014, pp. 1056–1061.
    27. 27)
      • 29. Ranzato, M., Hinton, G.E.: ‘Modeling pixel means and covariances using factorized third-order Boltzmann machines’. Proc. IEEE Int. Conf. Computer Vision and Pattern Recognition, San Francisco, USA, June 2010, pp. 2551–2558.
    28. 28)
      • 5. Wang, W., Huang, Y., Wang, Y.Z., et al: ‘Generalized autoencoder: a neural network framework for dimensionality reduction reducing the dimensionality of data with neural networks’. Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, June 2014, pp. 496–503.
    29. 29)
      • 6. Cheng, J., Liu, H.J., Wang, F., et al: ‘Silhouette analysis for human action recognition based on supervised temporal t-SNE and incremental learning’, IEEE Trans. Image Process., 2015, 24, (10), pp. 3203–3217.
    30. 30)
      • 4. Zhang, W., Zhou, W.J., Li, B.: ‘Fault diagnosis approach based on fractal dimension LLE and Fisher discrimination’, Chin. J. Sci. Instrum., 2010, 3, (2), pp. 325–333.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2015.0482
Loading

Related content

content/journals/10.1049/iet-cvi.2015.0482
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading