Your browser does not support JavaScript!
http://iet.metastore.ingenta.com
1887

access icon free New fusional framework combining sparse selection and clustering for key frame extraction

Key frame extraction can facilitate rapid browsing and efficient video indexing in many applications. However, to be effective, key frames must preserve sufficient video content while also being compact and representative. This study proposes a syncretic key frame extraction framework that combines sparse selection (SS) and mutual information-based agglomerative hierarchical clustering (MIAHC) to generate effective video summaries. In the proposed framework, the SS algorithm is first applied to the original video sequences to obtain optimal key frames. Then, using content-loss minimisation and representativeness ranking, several candidate key frames are efficiently selected and grouped as initial clusters. A post-processor – an improved MIAHC – subsequently performs further processing to eliminate redundant images and generate the final key frames. The proposed framework overcomes issues such as information redundancy and computational complexity that afflict conventional SS methods by first obtaining candidate key frames instead of accurate key frames. Subsequently, application of the improved MIAHC to these candidate key frames rather than the original video not only results in the generation of accurate key frames, but also reduces the computation time for clustering large videos. The results of comparative experiments conducted on two benchmark datasets verify that the performance of the proposed SS–MIAHC framework is superior to that of conventional methods.

References

    1. 1)
      • 24. Truong, B.T., Vemkatesh, S.: ‘Video abstraction: a systematic review and classification’, ACM Trans. Multimedia Comput. Commun. Appl. (TOMM), 2007, 3, (1), p. 3.
    2. 2)
      • 7. Potapov, D., Douze, M., Harchaoui, Z., et al: ‘Category-specific video summarization’. Proc. of ECCV, 2014, pp. 540555.
    3. 3)
      • 10. Li, L., Zhou, K., Xue, G.R., et al: ‘Video summarization via transferrable structured learning’, in Srinivasan, S., Ramamritham, K., Kumar, A., Ravindra, M.P., Bertino, E., Kumar, R. (Eds.): ‘WWW’ (ACM, 2011), pp. 287296.
    4. 4)
      • 17. Jiang, R.M., Leigh, A.H., Crooks, D.: ‘Advances in video summarization and skimming’, in Grgic, M., Delac, K., Ghanbari, M. (Eds.): ‘Recent advances in multimedia signal processing and communication’ (Springer, Berlin, Heidelberg, 2009, 1st edn.), pp. 2750.
    5. 5)
      • 8. Zhao, B., Xing, E.P.: ‘Quasi real-time summarization for consumer videos’. Proc. of CVPR, 2013, pp. 25132520.
    6. 6)
      • 35. ‘VSUMM (Video Summarization)’. Available at https://sites.google.com/site/vsummsite/, accessed 18 January 2015.
    7. 7)
      • 5. Pal, S.K., Leigh, A.B.: ‘Motion frame analysis and scene abstraction: discrimination ability of fuzziness measures’, J. Intell. Fuzzy Syst., 1995, 3, (3), pp. 247256.
    8. 8)
      • 14. Gygli, M., Grabner, H., Riemenschneider, H.: ‘Creating summaries from user videos’. Proc. of ECCV, 2014, pp. 505520.
    9. 9)
      • 34. Stricker, M., Orengo, M.: ‘Similarity of color images’, Proc. SPIE, 1995, 2420, pp. 381392.
    10. 10)
      • 27. Li, Z., Schuster, G.M., Katsaggelos, A.K., et al: ‘MINMAX optimal video summarization’, IEEE Trans. Circuits Syst. Video Technol., 2005, 15, (10), pp. 12451256.
    11. 11)
      • 20. Liu, Y.L., Xiao, Y.: ‘A robust image hashing algorithm resistant against geometrical attacks’, Radio Eng., 2013, 22, (4), pp. 10721081.
    12. 12)
      • 30. Vila, M., Bardera, A., Xu, Q., et al: ‘Tsallis entropy-based information measures for shot boundary detection and keyframe selection’, Signal Image & Video Processing (SIVIP), 2013, 7, (3), pp. 507520.
    13. 13)
      • 28. Cong, Y., Yuan, J.S., Luo, J.B.: ‘Towards scalable summarization of consumer videos via sparse dictionary selection’, IEEE Trans. Multimedia, 2012, 14, (1), pp. 6675.
    14. 14)
      • 1. Wang, M., Hong, R., Li, G., et al: ‘Event driven web video summarization by tag localization and key-shot identification’, IEEE Trans. Multimed., 2012, 14, (4), pp. 975985.
    15. 15)
      • 3. Zhao, L., Qi, W., Li, S.Z.: ‘Key-frame extraction and shot retrieval using nearest feature line (NFL)’. Proc. of 2000 ACM Workshops on Multimedia, New York, 2000, pp. 217220.
    16. 16)
      • 33. Wu, J., Rehg, J.M.: ‘Centrist: a visual descriptor for scene categorization’, IEEE Trans. Pattern Anal. Mach. Intell., 2011, 33, (8), pp. 14891501.
    17. 17)
      • 32. Nesterov, Y.: ‘Gradient method for minimizing composite objective function’ (CORE, Louvain-la-Neuve, Belgium, 2007).
    18. 18)
      • 13. Kumar, M., Loui, A.C.: ‘Key frame extraction from consumer videos using sparse representation’. Proc. of 18th IEEE ICIP, Brussels, September 2011, pp. 24372440.
    19. 19)
      • 6. Avila, S.E.F., Lopes, A.B.P., Antonio, L.J., et al: ‘VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method’, Pattern Recognit. Lett., 2011, 32, (1), pp. 5668.
    20. 20)
      • 19. Furini, M., Geraci, F., Montangero, M., et al: ‘STIMO: still and moving video storyboard for the web scenario’, Multimedia Tools Appl., 2010, 46, (1), pp. 4749.
    21. 21)
      • 26. DeMenthon, D., Kobla, V., Doermann, D.: ‘Video summarization by curve simplification’. Proc. of ACM Int. Conf. on Multimedia, New York, 1998, pp. 211218.
    22. 22)
      • 25. Money, A., Agius, H.: ‘Video summarization: a conceptual framework and survey of the state of the art’, J. Vis. Commun. Image Represent., 2008, 19, (2), pp. 121143.
    23. 23)
      • 16. Fu, Y., Liu, H., Cheng, Y.: ‘Key frame selection in WCE video based on shot detection’. 10th WCICA, Beijing, July 2012, pp. 50305034.
    24. 24)
      • 4. Cernekova, Z., Pitas, I., Nikou, C.: ‘Information theory-based shot cut/fade detection and video summarization’, IEEE Trans. Circuits Syst. Video Technol., 2006, 16, (1), pp. 8291.
    25. 25)
      • 2. Yarmohammadi, H., Rahmati, M., Khadivi, S.: ‘Content based video retrieval using information theory’. Proc. of Iranian Conf. on Machine Vision and Image Processing, Zanjan, September 2013, pp. 214218.
    26. 26)
      • 29. Mei, S.H., Guan, G.L., Wang, Z.Y., et al: ‘Video summarization via minimum sparse reconstruction’, Pattern Recognit., 2015, 48, (2), pp. 552533.
    27. 27)
      • 23. Panin, G., Knoll, A.: ‘Mutual information-based 3D object tracking’, Int. J. Comput. Vis., 2008, 78, (1), pp. 107118.
    28. 28)
      • 9. Lienhart, R., Pfeiffer, S., Effelsberg, W.: ‘Video abstracting’, Commun. ACM, 1997, 40, (12), pp. 5562.
    29. 29)
      • 22. Cernekova, Z., Nikou, C., Pitas, I.: ‘Shot detection in video sequences using entropy-based metrics’. Proc. of Int. Conf. on Image Processing, July 2002, vol. 3, pp. 421424.
    30. 30)
      • 21. Vretos, N., Solachidis, V., Pitas, I.: ‘A mutual information based face clustering algorithm for movie content analysis’, Image Vis. Comput., 2011, 29, (10), pp. 693705.
    31. 31)
      • 18. Mundur, P., Yesha, Y.: ‘Keyframe-based video summarization using Delaunay clustering’, Int. J. Digit. Libr., 2006, 6, (2), pp. 219232.
    32. 32)
      • 15. Ejaz, N., Mehmood, M., Baik, S.W.: ‘Feature aggregation based visual attention model for video summarization’, Comput. Electr. Eng., 2014, 40, (3), pp. 9931005.
    33. 33)
      • 31. Fu, Y., Guo, Y., Zhu, Y., et al: ‘Multi-view video summarization’, IEEE Trans. Multimedia, 2010, 12, (7), pp. 717729.
    34. 34)
      • 11. Baber, J., Afzulpurkar, N., Dailey, M., et al: ‘Shot boundary detection from videos using entropy and local descriptor’. Proc. of Int. Conf. on Digital Signal, Corfu, July 2011, pp. 16.
    35. 35)
      • 12. Zhang, Q., Yu, S.P.: ‘An efficient method of key-frame extraction based on a cluster algorithm’, J. Human Kinetics, 2013, 39, (1), pp. 513.
http://iet.metastore.ingenta.com/content/journals/10.1049/iet-cvi.2015.0237
Loading

Related content

content/journals/10.1049/iet-cvi.2015.0237
pub_keyword,iet_inspecKeyword,pub_concept
6
6
Loading
This is a required field
Please enter a valid email address