Recognition of Advertisement Emotions With Application to Computational Advertising

Abstract

Advertisements (ads) often contain strong affective content to capture viewer attention and convey an effective message to the audience. However, most computational affect recognition (AR) approaches examine ads via the text modality, and only limited work has been devoted to decoding ad emotions from audiovisual or user cues. This work (1) compiles an affective ad dataset capable of evoking coherent emotions across users; (2) explores the efficacy of content-centric convolutional neural network (CNN) features for AR vis-ã-vis handcrafted audio-visual descriptors; (3) examines user-centric ad AR from Electroencephalogram (EEG) responses acquired during ad-viewing, and (4) demonstrates how better affect predictions facilitate effective computational advertising as determined by a study involving 18 users. Experiments reveal that (a) CNN features outperform audiovisual descriptors for content-centric AR; (b) EEG features are able to encode ad-induced emotions better than content-based features; (c) Multi-task learning performs best among a slew of classification algorithms to achieve optimal AR, and (d) Pursuant to (b), EEG features also enable optimized ad insertion onto streamed video, as compared to contentbased or manual insertion techniques in terms of ad memorability and overall user experience.

Automated Parkinson's Disease Detection and Affective Analysis from Emotional EEG Signals

“…Deep neural networks are the state-of-the-art in text, speech, image, video and EEG-based recognition [48], [65], [49], [47], [25], [21], and have outperformed traditional machine learning methods obviating the need for handcrafted features [62]. We explored 1D, 2D and 3D-CNNs to learn EEG representations.…”

“…Implicit physiological or biosignals reflect characteristic activity of the central nervous system, and cannot be intentionally suppressed. Recent studies have extensively employed biosignals [19], [20], [21] for emotion perception in healthy subjects. EEG, functional Magnetic Resonance Imaging (fMRI), Magnetoencephalogram (MEG) and Positron Emission Tomography (PET) provide reliable information on emotional states compared to other modalities [22].…”

“…EEG frequency bands are known to correlate with emotions [23], [24]. Handcoded EEG descriptors such as Spectral Power Vectors enable emotion detection while Convolutional Neural Networks (CNNs) can automatically learn cognitive and emotional correlates [25], [21].…”

Does Visual Self-Supervision Improve Learning of Speech Representations for Emotion Recognition?

“…Audiovisual emotion recognition has also seen a significant amount of recent research efforts. Automatic affect recognition has a variety of applications in various fields; from detecting depression [37], to more emotionally relevant advertising [38], [39]. A lot of contemporary affect analysis approaches are based on deep neural networks that study both the visual and audio modalities [40], [41].…”

The eyes know it: FakeET -- An Eye-tracking Database to Understand Deepfake Perception

“…Table 1 summarizes statistics of the FakeET dataset, in terms of the eye-gaze and EEG recordings available for research. While a substantial amount of EEG data was visually found to be noisy, and ignored for our analyses, we nevertheless note that sophisticated machine learning algorithms designed for noisy data (e.g., multiple instance learning, Siamese neural networks) can be explored to improve EEG-based FD as in [18]. In terms of eye-gaze recordings, each video corresponds to a minimum of 2, and a maximum of 16 eye-track files (some data was lost due to subjects closing their eyes, tracker errors, etc.).…”