Automatic highlights extraction for drama video using music emotion and human face features

Lin, Keng-Sheng; Lee, Ann; Yang, Yi-Hsuan; Lee, Cheng-Te; Chen, Homer H.

doi:10.1016/j.neucom.2012.03.034

Cited by 14 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A system using music emotion and the human face as features for drama video has been proposed. 13 The method concerned uses two high-level features (music emotion and the human face) and two low-level features (shot duration and motion magnitude) to extract highlights. The authors claim that the method is effective for video highlight extraction.…”

Section: Related Workmentioning

confidence: 99%

Classification of Music Mood Using MPEG-7 Audio Features and SVM with Confidence Interval

Sarno

Ridoean

Sunaryono

et al. 2018

Int. J. Artif. Intell. Tools

View full text Add to dashboard Cite

Psychologically, music can affect human mood and influence human behavior. In this paper, a novel method for music mood classification is introduced. In the experiment, music mood classification was performed using feature extraction based on MPEG-7 features from the ISO/IEC 15938 standard for describing multimedia content. The result of this feature extraction are 17 low-level descriptors. Here, we used the Audio Power, Audio Harmonicity, and Audio Spectrum Projection features. Moreover, the discrete wavelet transform (DWT) was utilized for audio signal reconstruction. The reconstructed audio signals were classified by the new method, which uses a support vector machine with a confidence interval (SVM-CI). According to the experimental results, the success rate of the proposed method was satisfactory and SVM-CI outperformed the ordinary SVM.

show abstract

Section: Related Workmentioning

confidence: 99%

Classification of Music Mood Using MPEG-7 Audio Features and SVM with Confidence Interval

Sarno

Ridoean

Sunaryono

et al. 2018

Int. J. Artif. Intell. Tools

View full text Add to dashboard Cite

show abstract

“…First, following the definition in previous work (M. Xu, Jin, Luo, & Duan, 2008), we define highlights as the most memorable shots in a video with high emotion intensity. Note that highlight detection is different from video summarization, which focuses on condensed storyline representation of a video, rather than extracting affective contents (K.-S. Lin, Lee, Yang, Lee, & Chen, 2013).…”

Section: Highlight Detection By Video Processingmentioning

confidence: 99%

“…For highlight detection, some researchers propose to represent emotions in a video by a curve on the arousal-valence plane with low-level features such as motion, vocal effects, shot length, and audio pitch (Hanjalic & Xu, 2005), color (Ngo, Ma, & Zhang, 2005), mid-level features such as laughing and subtitles (M. Xu, Luo, Jin, & Park, 2009). Nevertheless, due to the semantic gap between low-level features and high-level semantics, accuracy of highlight detection based on video processing is limited (K.-S. Lin et al, 2013).…”

Section: Highlight Detection By Video Processingmentioning

confidence: 99%

Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowdsourced Time-Sync Comments

Ping¹,

Chen²

2017

Proceedings of the Workshop on New Frontiers in Summarization

View full text Add to dashboard Cite

With the prevalence of video sharing, there are increasing demands for automatic video digestion such as highlight detection. Recently, platforms with crowdsourced time-sync video comments have emerged worldwide, providing a good opportunity for highlight detection. However, this task is non-trivial: (1) time-sync comments often lag behind their corresponding shot; (2) time-sync comments are semantically sparse and noisy; (3) to determine which shots are highlights is highly subjective. The present paper aims to tackle these challenges by proposing a framework that (1) uses concept-mapped lexical-chains for lagcalibration; (2) models video highlights based on comment intensity and combination of emotion and concept concentration of each shot; (3) summarize each detected highlight using improved SumBasic with emotion and concept mapping. Experiments on large real-world datasets show that our highlight detection method and summarization method both outperform other benchmarks with considerable margins.

show abstract

“…Moreover, other studies rely on approaches that are based on the exploitation of audio by detecting any sudden variation of sound recording [8]- [10] caused by audience reaction, such as applauds or shouts out, this variation reflect the highlight moment during the event (Fig. 2), but unfortunately this approach can have several inconveniences such as special effects added during the event, or even the presence of noise during the whole event.…”

Section: Introductionmentioning

confidence: 99%

Generating a Highlight Moments Summary Video of Apolitical Event using Ontological Analysis on Social Media Speech Sentiment

Mehdi¹,

Nsiri²,

Serhane³

et al. 2019

ijacsa

View full text Add to dashboard Cite

Numerous viewers choose to watch political or presidential debates highlights via TV or internet, rather than seeing the whole debate nowadays, which requires a lot of time. However, the task of making a debate summary, which can be considered neutral and does not give out a negative nor a positive image of the speaker, has never been an easy one, due to personal or political beliefs bias of the video maker. This study came up with a solution that generates highlights of a political event, based on twitter social network flow. Twitter streaming API is used to detect an event's tweets stream using specific hashtags, and detect on a timescale the extreme changes of volume of tweets, which will determine the highlight moments of our video summary at first, then a process is set up based on a group of ontologies that analyze each tweet of these moments to calculate the percentage of each sentiment's positivity, then classify those moments by category (positive, negative or neutral).

show abstract

Automatic highlights extraction for drama video using music emotion and human face features

Cited by 14 publications

References 16 publications

Classification of Music Mood Using MPEG-7 Audio Features and SVM with Confidence Interval

Classification of Music Mood Using MPEG-7 Audio Features and SVM with Confidence Interval

Video Highlights Detection and Summarization with Lag-Calibration based on Concept-Emotion Mapping of Crowdsourced Time-Sync Comments

Generating a Highlight Moments Summary Video of Apolitical Event using Ontological Analysis on Social Media Speech Sentiment

Contact Info

Product

Resources

About