EmotionMap: Visual Analysis of Video Emotional Content on a Map

Ma, Chenyue; Song, Jiancheng; Zhu, Qian; Maher, Kevin; Huang, Zeyuan; Wang, Hongan

doi:10.1007/s11390-020-0271-2

Cited by 14 publications

(6 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…EmotionCues [25] summarized the emotional dynamics in classroom videos and highlighted model uncertainties by a stream graph design. EmotionMap [26] and E-ffective [29] proposed a map-based and spiral-based design to provide a temporal overview of the affective dimension in multimedia videos. These visualizations effectively presented visual cues for notable moments.…”

Section: Visual Analysis Of Multimodal Videosmentioning

confidence: 99%

Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events

Kwai¹,

Wang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

Delivering customer services through video communications has brought new opportunities to analyze customer satisfaction for quality management. However, due to the lack of reliable self-reported responses, service providers are troubled by the inadequate estimation of customer services and the tedious investigation into multimodal video recordings. We introduce Anchorage, a visual analytics system to evaluate customer satisfaction by summarizing multimodal behavioral features in customer service videos and revealing abnormal operations in the service process. We leverage the semantically meaningful operations to introduce structured event understanding into videos which help service providers quickly navigate to events of their interest. Anchorage supports a comprehensive evaluation of customer satisfaction from the service and operation levels and efficient analysis of customer behavioral dynamics via multifaceted visualization views. We extensively evaluate Anchorage through a case study and a carefully-designed user study. The results demonstrate its effectiveness and usability in assessing customer satisfaction using customer service videos. We found that introducing event contexts in assessing customer satisfaction can enhance its performance without compromising annotation precision. Our approach can be adapted in situations where unlabelled and unstructured videos are collected along with sequential records.

show abstract

Section: Visual Analysis Of Multimodal Videosmentioning

confidence: 99%

Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events

Kwai¹,

Wang²,

Wang³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The size of the text was also changed in order to highlight emotionally intense moments in the speech. The method of mapping of the color to the valence and arousal in the speech script can be seen in the work [21].…”

Section: E-scriptmentioning

confidence: 99%

E-ffective: A Visual Analytic System for Exploring the Emotion and Effectiveness of Inspirational Speeches

Maher

Huang²,

Song

et al. 2022

IEEE Trans. Visual. Comput. Graphics

Self Cite

View full text Add to dashboard Cite

What makes speeches effective has long been a subject for debate, and until today there is broad controversy among public speaking experts about what factors make a speech effective as well as the roles of these factors in speeches. Moreover, there is a lack of quantitative analysis methods to help understand effective speaking strategies. In this paper, we propose E-ffective, a visual analytic system allowing speaking experts and novices to analyze both the role of speech factors and their contribution in effective speeches. From interviews with domain experts and investigating existing literature, we identified important factors to consider in inspirational speeches. We obtained the generated factors from multi-modal data that were then related to effectiveness data. Our system supports rapid understanding of critical factors in inspirational speeches, including the influence of emotions by means of novel visualization methods and interaction. Two novel visualizations include E-spiral (that shows the emotional shifts in speeches in a visually compact way) and E-script (that connects speech content with key speech delivery information). In our evaluation we studied the influence of our system on experts' domain knowledge about speech factors. We further studied the usability of the system by speaking novices and experts on assisting analysis of inspirational speech effectiveness.

show abstract

“…Visual video analytics refers to the reasoning process facilitated by a visualization interface to complete video analytic tasks [13,33]. Various visual analytic approaches have been developed for movies [40,44], news [36,53], presentations [66,77], and education [22,76].…”

Section: Visual Video Analyticsmentioning

confidence: 99%

“…To analyze such information, Kurzhals et al [40] proposed multilayer segmented timelines to characterize the visual content of movies and associated video frames with semantic information derived from subtitles or scripts. Meanwhile, Ma et al [44] focused on the emotional content of movies and employed a map metaphor for tracking and understanding the evolution of characters' emotions. News videos are important for journalists to discover significant topics and emergent events.…”

Section: Visual Video Analyticsmentioning

confidence: 99%

See 1 more Smart Citation

VideoModerator: A Risk-aware Framework for Multimodal Video Moderation in E-Commerce

Tang¹,

Wu²,

Yu³

et al. 2021

Preprint

View full text Add to dashboard Cite

Video moderation, which refers to remove deviant or explicit content from e-commerce livestreams, has become prevalent owing to social and engaging features. However, this task is tedious and time consuming due to the difficulties associated with watching and reviewing multimodal video content, including video frames and audio clips. To ensure effective video moderation, we propose VideoModerator, a risk-aware framework that seamlessly integrates human knowledge with machine insights. This framework incorporates a set of advanced machine learning models to extract the risk-aware features from multimodal video content and discover potentially deviant videos. Moreover, this framework introduces an interactive visualization interface with three views, namely, a video view, a frame view, and an audio view. In the video view, we adopt a segmented timeline and highlight high-risk periods that may contain deviant information. In the frame view, we present a novel visual summarization method that combines risk-aware features and video context to enable quick video navigation. In the audio view, we employ a storyline-based design to provide a multi-faceted overview which can be used to explore audio content. Furthermore, we report the usage of VideoModerator through a case scenario and conduct experiments and a controlled user study to validate its effectiveness.

show abstract

EmotionMap: Visual Analysis of Video Emotional Content on a Map

Cited by 14 publications

References 45 publications

Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events

Anchorage: Visual Analysis of Satisfaction in Customer Service Videos via Anchor Events

E-ffective: A Visual Analytic System for Exploring the Emotion and Effectiveness of Inspirational Speeches

VideoModerator: A Risk-aware Framework for Multimodal Video Moderation in E-Commerce

Contact Info

Product

Resources

About