Saliency detection has been widely studied to predict human fixations, with various applications in wireless multimedia communications. For saliency detection, we argue that the state-of-the-art high-efficiency video-coding (HEVC) standard can be used to generate the useful features in compressed domain. Therefore, this chapter proposes to learn the video-saliency model, with regard to HEVC features. First, we establish an eye-tracking database for video-saliency detection. Through the statistical analysis on our eye-tracking database, we find out that human fixations tend to fall into the regions with large-valued HEVC features on splitting depth, bit allocation, and motion vector (MV). In addition, three observations are obtained from the further analysis on our eyetracking database. Accordingly, several features in HEVC domain are proposed on the basis of splitting depth, bit allocation, and MV. Next, a support vector machine (SVM) is learned to integrate those HEVC features together, for video-saliency detection. Since almost all video data are stored in the compressed form, our method is able to avoid both the computational cost on decoding and the storage cost on raw data. More importantly, experimental results show that the proposed method is superior to other state-of-the-art saliency-detection methods, either in compressed or uncompressed domain.

Chapter Contents:

• 9.1 Introduction
• 9.2 Related work on video-saliency detection
• 9.2.1 Heuristic video-saliency detection
• 9.2.2 Data-driven video-saliency detection
• 9.3 Database and analysis
• 9.3.1 Database of eye tracking on raw videos
• 9.3.2 Analysis on our eye-tracking database
• 9.3.3 Observations from our eye-tracking database
• 9.4 HEVC features for saliency detection
• 9.4.1 Basic HEVC features
• 9.4.2 Temporal difference features in HEVC domain
• 9.4.3 Spatial difference features in HEVC domain
• 9.5 Machine-learning-based video-saliency detection
• 9.5.1 Training algorithm
• 9.5.2 Saliency detection
• 9.6 Experimental results
• 9.6.1 Setting on encoding and training
• 9.6.2 Analysis on parameter selection
• 9.6.3 Evaluation on our database
• 9.6.4 Evaluation on other databases
• 9.6.5 Evaluation on other work conditions
• 9.6.6 Effectiveness of single features and learning algorithm
• 9.7 Conclusion
• References

Inspec keywords: feature extraction; statistical analysis; video signal processing; learning (artificial intelligence); video coding; object detection; multimedia communication; support vector machines; data compression

Other keywords: state-of-the-art high-efficiency video-coding; wireless multimedia communications; state-of-the-art saliency-detection methods; video decoding application; large-valued HEVC features; video data; support vector machine; bit allocation; video-saliency detection; machine-learning-based saliency detection; video-saliency model; splitting depth; HEVC domain; useful features; eye-tracking database; human fixations; eyetracking database

Subjects: Computer vision and image processing techniques; Knowledge engineering techniques; Image and video coding; Video signal processing; Optical, image and video signal processing; Multimedia communications