Movie abstraction via the progress of the storyline

Author(s): S. Zhu ; Y. Zhao ; Z. Liang ; X. Jing
DOI: 10.1049/iet-spr.2011.0182

For access to this article, please select a purchase option:

Buy article PDF

Buy Knowledge Pack

IET members benefit from discounts to all IET publications and free access to E&T Magazine. If you are an IET member, log in to your account and the discounts will automatically be applied.

Learn more about IET membership

Recommend Title Publication to library

IET Signal Processing — Recommend this title to your library

Thank you

Your recommendation has been sent to your librarian.

Author(s): S. Zhu ¹ ; Y. Zhao ² ; Z. Liang ¹ ; X. Jing ¹
- Affiliations: 1: School of Automation, Nanjing University of Post and Telecommunications, Nanjing, People's Republic of China
  2: College of Mechanical and Electronic Engineering, Nanjing Forestry University, Nanjing, People's Republic of China
Source: Volume 6, Issue 8, October 2012, p. 751 – 762
DOI: 10.1049/iet-spr.2011.0182 , Print ISSN 1751-9675, Online ISSN 1751-9683

An appropriate movie abstraction is helpful for movie producers to promote the progress of the storyline as well as for the audiences to capture the theme of the movie before watching the full-length movie. Most existing movie abstraction schemes rely heavily on video content only, which may not deliver ideal results because of the semantic gap between computer calculated low-level audiovisual features and human used high-level perceptual understanding. In this study, the authors incorporate script into movie content understanding and present a new movie abstraction approach via the progress of the storyline, which is the soul of a film that actually catches the audiences' attention. The authors first segment the movie scenes by analysis of the movie script. Then the authors conduct storyline analysis using the attention analysis and audiovisual features. Given the transition intensity values, the authors calculate the storyline progress score and adopt this as the criterion to generate movie abstraction. The promising experimental results demonstrate that the analysis of storyline evolution is an effective approach for the abstraction and understanding of movie content.

References

1. 1)
  - Bordwell, D., Thompson, K.: `Film art: an introduction', Technology Report, 1996.
2. 2)
  - Ma, Y., Lu, L., Zhang, H., Li, M.: `A user attention model for video summarization', Proc. ACM Int. Conf. Multimedia, 2002, p. 533–542.
3. 3)
  - C. Weng , W. Chu , J. Wu . Rolenet: movie analysis from the perspective of social networks. IEEE Trans. Multimedia , 2 , 256 - 271
4. 4)
  - Y. Gao , W. Wang , J. Yong , H. Gu . Dynamic video summarization using two-level redundancy detection. Multimedia Tools Appl. , 2 , 233 - 250
5. 5)
  - Li, K., Guo, L., Faraco, C.: `Human-centered attention models for video summarization', Proc. IEEE Int. Conf. Multimodal Interfaces, 2010, p. 27–30.
6. 6)
  - C. Ngo , Y. Ma , H. Zhang . Video summarization and scene detection by graph modeling. IEEE Trans. Circuits Syst. Video Technol. , 2 , 296 - 305
7. 7)
  - S. Zhu , Y. Liu . Two-dimensional entropy model for video shot partitioning. Sci. China F, Inf. Sci. , 2 , 183 - 194
8. 8)
  - D. Besiris , A. Makedonas , G. Economou . Spiros fotopoulos: combining graph connectivity and dominant set clustering for video summarization. Multimedia Tools Appl. , 2 , 161 - 186
9. 9)
  - S. Zhu , Y. Liu . Automatic scene detection for advanced story retrieval. Expert Syst. Appl. , 3 , 5976 - 5986
10. 10)
  - Li, S., Zhu, L., Zhang, Z., Blake, A., Zhang, H., Shum, H.: `Statistic learning of multi-view face detection', Proc. IEEE Int. Conf. Computer Vision, 2002, p. 67–81.
11. 11)
  - Y. Li , C. Kuo . (2003) Video content analysis using multimodal in formation: for movie content extraction, indexing and representation.
12. 12)
  - Evangelopoulos, G., Rapantzikos, K., Potamianos, A.: `Movie summarization based on audiovisual saliency detection', Proc. IEEE Int. Conf. Image Processing, 2008, p. 2528–2531.
13. 13)
  - B. Yeo , B. Liu . Rapid scene analysis on compressed video. IEEE Trans. Circuits Syst. Video Technol. , 6 , 533 - 544
14. 14)
  - J. Monaco . (1982) How to read a film: the art, technology, language, history and theory of film and media.
15. 15)
  - Y. Li , S. Lee , C. Yeh , C. Kuo . Techniques for movie content analysis and skimming. IEEE Signal Process. Mag. , 79 - 89
16. 16)
  - Y. Ma , X. Hua , L. Lu , H. Zhang . A generic framework of user attention model and its application in video summarization. IEEE Trans. Multimedia , 5 , 907 - 919
17. 17)
  - L. Bai , Y. Hu , S. Lao . O'Connor: automatic summarization of rushes video using bipartite graphs. Multimedia Tools Appl. , 1 , 63 - 80
18. 18)
  - Lu, S., King, I., Lyu, M.: `Video summarization by video structure analysis and graph optimization', Proc. IEEE Int. Conf. Multimedia and Expo, 2004, p. 1959–1962.
19. 19)
  - Tang, L., Mei, T., Hua, X.: `Near-lossless video summarization', Proc. ACM Int. Conf. Multimedia, 2009, p. 351–360.
20. 20)
  - H. Sundaram , S. Chang . Computable scenes and structures in films. IEEE Trans. Multimedia , 4 , 482 - 491

Movie abstraction via the progress of the storyline

Movie abstraction via the progress of the storyline

Buy article PDF

Buy Knowledge Pack

Thank you

References

Related content