access icon free Multi-instance multi-label learning of natural scene images: via sparse coding and multi-layer neural network

© The Institution of Engineering and Technology
Received 27/09/2016, Accepted 03/11/2017, Revised 24/09/2017, Published 16/11/2017

The classification of natural scene images is multi-instance multi-label (MIML) for many labels that exist in a natural scene image. The traditional method of solving MIML is to degenerate it into single-instance single-label learning (SISL). However, the precision of the method could decrease due to information loss during the degeneration process. How to reasonably solve the MIML problem is key to obtaining high accuracy in this research area. An MIML algorithm based on instances via combining sparse coding with a deep neural network is proposed. First, an instance-based sparse representation with dictionary learning is adopted. Second, an MIML description model based on a deep network is proposed, which can realise parameter self-learning in combination with sparse representations. Third, the residuals of the sparse representation are introduced to the deep neural network. The results of the experiments show that the method outperforms a number of state-of-the-art approaches.

Inspec keywords: learning (artificial intelligence); multilayer perceptrons; image coding; image representation

Other keywords: information loss; deep neural network; MIML description model; instance-based sparse representation; multiinstance multilabel learning; natural scene images; degeneration process; parameter self-learning; MIML algorithm; sparse coding; multilayer neural network; dictionary learning

Subjects: Computer vision and image processing techniques; Neural computing techniques; Image and video coding; Knowledge engineering techniques

References

    1. 1)
      • 24. Schapire, R.E., Singer, Y.: ‘BoosTexter: a boosting-based system for text categorization’, Mach. Learn, 2000, 39, (2–3), pp. 135–168.
    2. 2)
      • 3. Dietterich, T.G., Lathrop, R.H., Lozano-Pérez, T.: ‘Solving the multiple instance problem with axis-parallel rectangles’, Artif. Intell., 1997, 89, pp. 31–71.
    3. 3)
      • 53. Triguero, I., Vens, C.: ‘Labelling strategies for hierarchical multi-label classification techniques’, Pattern Recognit.., 2015, 56, pp. 170–183.
    4. 4)
      • 6. Zhang, Q., Goldman, S.A.: ‘EM-DD: an improved multiple-instance learning technique’, NIPS, 2002, 14, pp. 1073–1080.
    5. 5)
      • 49. Glorot, X., Bengio, Y.: ‘Understanding the difficulty of training deep feedforward neural networks’. Proc. 13th Int. Conf. Artificial Intelligence and Statistics, 2010, vol. 9, pp. 249–256.
    6. 6)
      • 20. Kwok, J.T., Cheung, P.M.: ‘Marginalized multi-instance kernels’. Int. Joint Conf. Artificial Intelligence (IJCAI), 2007, pp. 901–906.
    7. 7)
      • 13. Zhang, M., Zhou, Z.: ‘Adapting RBF neural networks to multi-instance learning’, Neural Process. Lett., 2006, 23, (1), pp. 1–26.
    8. 8)
      • 29. Elisseeff, A.: ‘Kernel methods for multi-labelled classification and categorical regression problems’, Adv. Neural Inf. Process., 2001, 2, pp. 681–687.
    9. 9)
      • 35. Zhang, M.L., Zhou, Z.H.: ‘Multi-instance clustering with applications to multi-instance prediction’, Appl. Intell., 2009, 31, (1), pp. 47–68.
    10. 10)
      • 23. Xu, X., Frank, E.: ‘Logistic regression and boosting for labeled bags of instances’. Proc. PacificAsia Conf. Knowledge Discovery and Data Mining, 2004, pp. 272–281.
    11. 11)
      • 42. Aharon, M., Elad, M., Bruckstein, A.: ‘K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation’, IEEE Trans. Signal Process., 2006, 54, (11), pp. 4311–4322.
    12. 12)
      • 37. Zhang, M.L.: ‘A k-nearest neighbor based multi-instance multi-label learning algorithm’. Proc. – Int. Conf. Tools with Artificial Intelligence (ICTAI), 2010, vol. 2, pp. 207–212.
    13. 13)
      • 12. Zhou, Z., Zhang, M.: ‘Neural networks for multi-instance learning’. Int. Conf. Intelligent Information Technologies, 2002, pp. 455–459.
    14. 14)
      • 45. Wang, S., Jiao, L., Yang, S., et al: ‘SAR image target recognition via complementary spatial pyramid coding’, Neurocomputing, 2016, 196, pp. 125–132.
    15. 15)
      • 46. Chen, Z., Chi, Z., Fu, H., et al: ‘Multi-instance multi-label image classification: a neural approach’, Neurocomputing, 2013, 99, pp. 298–306.
    16. 16)
      • 34. Zhang, Y., Zhou, Z.-H.: ‘Multilabel dimensionality reduction via dependence maximization’, ACM Trans. Knowl. Discov. Data, 2010, 4, (3), pp. 1–21.
    17. 17)
      • 48. Zhang, M., Wang, Z.: ‘MimlRbf: RBF neural networks for multi-instance multi-label learning’, Neurocomputing., 2009, 72, (16), pp. 3951–3956.
    18. 18)
      • 17. Zhou, Z.: ‘On the relation between multi-instance learning and semi-supervised learning’, 2007, (1997).
    19. 19)
      • 7. Mason, L., Baxter, J., Bartlett, P., et al: ‘Boosting algorithms as gradient descent in function space’, Adv. Neural Inf. Process. Syst., 1999, 12, (April), pp. 512–518.
    20. 20)
      • 1. Zhou, Z., Zhang, M.: ‘Multi-instance multi-label learning with application to scene classification’, Adv. Neural Inf. Process. Syst., 2007, 19, pp. 1609–1616.
    21. 21)
      • 18. Gärtner, T., Flach, P.A., Kowalczyk, A., et al: ‘Multi-instance kernels’. Proc. Nineteenth Int. Conf. Machine Learning, 2002, (Mi), pp. 179–186.
    22. 22)
      • 41. Zhou, Z.-H., Zhang, M., Chen, K.-J.: ‘A novel bag generator for image database retrieval with multi-instance learning techniques’. Proc. Int. Conf. Tools with Artificial Intelligence, 2003, pp. 565–569.
    23. 23)
      • 14. Chevaleyre, Y., Zucker, J.-D.: ‘A framework for learning rules from multiple instance data’, 2003, pp. 49–60.
    24. 24)
      • 25. Chen, Y., Bi, J., Wang, J.Z., et al: ‘MILES: multiple-instance learning via embedded instance selection’, IEEE Trans. Pattern Anal. Mach. Intell., 2006, 28, (12), pp. 1–18.
    25. 25)
      • 47. Wu, J.S., Huang, S.J., Zhou, Z.H.: ‘Genome-wide protein function prediction through multi-instance multi-label learning’, IEEE/ACM Trans. Comput. Biol. Bioinform., 2014, 11, (5), pp. 891–902.
    26. 26)
      • 8. Wang, J., Zucker, J.-D.: ‘Solving multiple-instance problem: a lazy learning approach’. Proc. 17th Int. Conf. Machine Learning, 2000, (1994), pp. 1119–1125.
    27. 27)
      • 32. Liu, Y., Jin, R., Yang, L.: ‘Semi-supervised multi-label learning by constrained non-negative matrix factorization’. Proc. Natl. Conf. Artificial Intelligence, 2006, vol. 21, (1), p. 421.
    28. 28)
      • 16. Chen, Y., Wang, J.Z.: ‘Image categorization by learning and reasoning with regions’, J. Mach. Learn. Res., 2004, 5, pp. 913–939.
    29. 29)
      • 22. Zhou, Z.-H., Zhang, M.: ‘Ensembles of multi-instance learners’, Lect. Notes Artif. Intell. (Subseries Lect. Notes Comput. Sci.), 2003, 2837, pp. 492–502.
    30. 30)
      • 11. Zhang, M.L., Zhou, Z.H.: ‘Improve multi-instance neural networks through feature selection’, Neural Process. Lett., 2004, 19, (1), pp. 1–10.
    31. 31)
      • 19. Cheung, P.-M., Kwok, J.T.: ‘A regularization framework for multiple-instance learning’, Proc. 23rd Int. Conf. Mach. Learn. ACM, 2006, 23, (148), pp. 193–200.
    32. 32)
      • 44. Song, X., Liu, Z., Yang, X., et al: ‘A parameterized fuzzy adaptive K-SVD approach for the multi-classes study of pursuit algorithms’, Neurocomputing, 2014, 123, pp. 131–139.
    33. 33)
      • 26. Boutell, M.R., Luo, J., Shen, X., et al: ‘Learning multi-label scene classification’, Pattern Recognit.., 2004, 37, (9), pp. 1757–1771.
    34. 34)
      • 30. Godbole, S., Sarawagi, S.: ‘Discriminative methods for multi-labeled classification’, Lect. Notes Comput. Sci., 2004, 3056, pp. 22–30.
    35. 35)
      • 10. Blockeel, H., Page, D., Srinivasan, A.: ‘Multi-instance tree learning’, Proc. 22nd Int. Conf. Machine Learning, 2005, pp. 57–64.
    36. 36)
      • 40. Kohonen, T.: ‘The self-organizing map’, Neurocomputing, 1998, 21, (1–3), pp. 1–6.
    37. 37)
      • 21. Fung, G., Dundar, M., Krishnapuram, B., et al: ‘Multiple instance learning for computer aided diagnosis’, 2006.
    38. 38)
      • 9. Xi, I., Bergadano, F.: ‘Learning single and multiple instance decision trees for computer security applications’ (2000, February).
    39. 39)
      • 36. Zhang, M.: ‘M3MIML: a maximum margin method for multi-instance multi-label learning’. IEEE/Int. Conf. Data Mining, 2008, pp. 688–697.
    40. 40)
      • 5. Maron, O., Ratan, A.L.: ‘Multiple-instance learning for natural scene classification’. Proc. 15th Int. Conf. Machine Learning, 1998, pp. 341–349.
    41. 41)
      • 28. Li, Y.-X., Ji, S., Kumar, S., et al: ‘Drosophila gene expression pattern annotation through multi-instance multi-label learning’, IEEE/ACM Trans. Comput. Biol. Bioinforma., 2012, 9, (1), pp. 98–112.
    42. 42)
      • 38. Yang, C., Lozano-Perez, T.: ‘Image database retrieval with multiple-instance learning techniques’. Proc. 16th Int. Conf. Data Engineering 2000, 2000, pp. 233–243.
    43. 43)
      • 15. Andrews, S., Hofmann, T., Tsochantaridis, I.: ‘Multiple instance learning with generalized support vector machines’, AAAI, 2002, 15, (2), pp. 561–568.
    44. 44)
      • 39. Jiang, Y., Zhou, Z. H.: ‘SOM ensemble-based image segmentation’, Neural Process. Lett., 2004, 20, (3), pp. 171–178.
    45. 45)
      • 50. Salton, G.: ‘Automatic text processing: the transformation’, Anal. Retr. Inf. Comput., 1989, 14, p. 15.
    46. 46)
      • 4. Zhang, M.-L., Zhang, K.: ‘Multi-label learning by exploiting label dependency’. Proc. 16th ACM SIGKDD Int. Conf. Knowledge discovery and data mining - KDD'10, 2010, pp. 999–1008.
    47. 47)
      • 51. Yang, Y., Pedersen, J.O.: ‘A comparative study on feature selection in text categorization’. Proc. 14th Int. Conf. Machine Learning, 1997, pp. 412–420.
    48. 48)
      • 2. Zhou, Z.H., Zhang, M.L., Huang, S.J., et al: ‘Multi-instance multi-label learning’, Artif. Intell., 2012, 176, (1), pp. 2291–2320.
    49. 49)
      • 52. Lahrache, S., El Qadi, A., El Ouazzani, R.: ‘Bag-of-features for image memorability evaluation’, IET Comput. Vis., 2016, 10, (6), pp. 577–584.
    50. 50)
      • 33. Yu, K., Yu, S., Tresp, V.: ‘Multi-label informed latent semantic indexing’. Proc. 28th Annual Int. ACM, 2005, pp. 258–265.
    51. 51)
      • 31. Cesa-Bianchi, N., Gentile, C., Zaniboni, L.: ‘Hierarchical classification: combining Bayes with SVM’. Proc. 23rd Int. Conf. Machine Learning – ICML ‘06, 2006, pp. 177–184.
    52. 52)
      • 27. Nguyen, N.: ‘A new SVM approach to multi-instance multi-label learning’. Proc. - IEEE Int. Conf. Data Mining (ICDM), 2010, pp. 384–392.
    53. 53)
      • 43. Sajjad, M., Mehmood, I., Baik, S.W.: ‘Sparse coded image super-resolution using K-SVD trained dictionary based on regularized orthogonal matching pursuit’, Biomed. Mater. Eng., 2015, 26, (September), pp. S1399–S1407.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2016.0338
Loading

Related content

content/journals/10.1049/iet-cvi.2016.0338
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading