Automatic Curation of Sports Highlights Using Multimodal Excitement Features

Merler, Michele; Mac, Khoi-Nguyen C.; Joshi, Dhiraj; Nguyen, Quan; Hammer, Stephen; Kent, John; Xiong, Jinjun; Minh, N.; Smith, J. R.; Feris, Rogério

doi:10.1109/tmm.2018.2876046

Cited by 55 publications

(31 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…How to explore the characteristics of the excellent players' techniques and tactics from the rich data? Providing a scientific basis for coaches and athletes has become one of the most important problems in tennis theory research [16,17].…”

Section: The Present Situation Of Research On the Technique And Tactimentioning

confidence: 99%

Analysis of Characteristics of Tennis Singles Matches Based on 5G and Data Mining Technology

Qin-sheng

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

The level of technical and tactical decision-making in a tennis game has a very important impact on the outcome of the game. How to discover the characteristics and rules of the game from a large amount of technical and tactical data, how to overcome the shortcomings of traditional statistical methods, and how to provide a scientific basis for correct decision-making are a top priority. Based on 5G and association analysis data mining theory, we established a data mining model for tennis technical offensive tactics and association rules and conducted specific case studies. It can calculate the maximization and distribution rate of certain technologies, also distinguish between the athlete’s gain and loss rate and the spatial position on the track, and use artificial statistical methods to cause errors and subjective participation. This solution provides objective and scientific decision support for this problem and is used in the decision-making of the landing point in tennis match technology and tactics. Experimental simulation shows that the data mining technology analysis system used for regional tennis matches is more concise, efficient, and accurate than traditional movie analysis methods.

show abstract

Section: The Present Situation Of Research On the Technique And Tactimentioning

confidence: 99%

Analysis of Characteristics of Tennis Singles Matches Based on 5G and Data Mining Technology

Qin-sheng

et al. 2021

Security and Communication Networks

View full text Add to dashboard Cite

show abstract

“…Both of these only use visual features from videos. In terms of multimodal processing, [14,15,16] have presented novel approaches to finding highlights in sport videos, while [17] uses a multimodal approach for highlight detection in movies. These domains are however, constrained.…”

Section: Related Workmentioning

confidence: 99%

Audiovisual Highlight Detection in Videos

Mundnich

Fenster

Khare

et al. 2021

ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

In this paper, we test the hypothesis that interesting events in unstructured videos are inherently audiovisual. We combine deep image representations for object recognition and scene understanding with representations from an audiovisual affect recognition model. To this set, we include content agnostic audio-visual synchrony representations and mel-frequency cepstral coefficients to capture other intrinsic properties of audio. These features are used in a modular supervised model. We present results from two experiments: efficacy study of single features on the task, and an ablation study where we leave one feature out at a time. For the video summarization task, our results indicate that the visual features carry most information, and including audiovisual features improves over visual-only information. To better study the task of highlight detection, we run a pilot experiment with highlights annotations for a small subset of video clips and fine-tune our best model on it. Results indicate that we can transfer knowledge from the video summarization task to a model trained specifically for the task of highlight detection.

show abstract

“…Indeed, in order to match the important plots in these videos, professional video producers usually employ sounds to create an atmosphere and convey emotional content. And multimodal methods have already been proven to be effective for video understanding [7,8,9]. Therefore emotional information from audio modality can convey a key message concerning what is happening in a scene.…”

Section: Highlighted Clipsmentioning

confidence: 99%

“…Park et al [12] concatenated audio and visual features to determine the importance of frames in short videos. Merler et al [9] utilized audio information, including the commentator's speech and the cheers from the crowd, to help to summarize the exciting moments of sports games. As essential parts of human voice and music, emotional features have already been applied to audio analysis [13], and we assume that these features contribute significantly in the domain of video analysis as well.…”

Section: Related Workmentioning

confidence: 99%

Detecting Highlighted Video Clips Through Emotion-Enhanced Audio-Visual Cues

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Recent years have witnessed the growing research interests in video highlight detection. Existing studies mainly focus on detecting highlights in user-generated videos with simple topics based on visual content. However, relying solely on visual features limits the ability of conventional methods to capture highlights for videos with more complicated semantics, like movies. Therefore, we propose to mine the emotional information in video sounds to enhance highlight detection. Specifically, we design a novel emotion-enhanced framework with multi-stage fusion to detect highlights for complex videos. Along this line, we first extract multi-grained features from the audio waves. Then, the tailored-designed intra-modal fusion is applied on audio features to obtain emotional representation. Furthermore, the cross-modal fusion is developed to generate comprehensive representation of clip by merging audio emotional representations and visual features. This representation can be leveraged for predicting highlight probability. Finally, extensive experiments on realworld datasets demonstrate the effectiveness of our method.

show abstract

Automatic Curation of Sports Highlights Using Multimodal Excitement Features

Cited by 55 publications

References 33 publications

Analysis of Characteristics of Tennis Singles Matches Based on 5G and Data Mining Technology

Analysis of Characteristics of Tennis Singles Matches Based on 5G and Data Mining Technology

Audiovisual Highlight Detection in Videos

Detecting Highlighted Video Clips Through Emotion-Enhanced Audio-Visual Cues

Contact Info

Product

Resources

About