Implicit Analysis of Perceptual Multimedia Experience Based on Physiological Response: A Review

Abstract

The exponential growth of popularity of multimedia has led needs for user-centric adaptive applications that manage multimedia content more effectively. Implicit analysis, which examines users' perceptual experience of multimedia by monitoring physiological or behavioral cues, has potential to satisfy such demands. Particularly, physiological signals categorized into cerebral physiological signals (electroencephalography, functional magnetic resonance imaging, and functional nearinfrared spectroscopy) and peripheral physiological signals (heart rate, respiration, skin temperature, etc.) have recently received attention along with notable development of wearable physiological sensors. In this paper, we review existing studies on physiological signal analysis exploring perceptual experience of multimedia. Furthermore, we discuss current trends and challenges.Index Terms-physiological signal, perceptual experience, implicit analysis, multimedia 1 Overviews of implicit measurement techniques using behavioral cues can be found in [6], [7].This is the author's version of an article that has been published in this journal. Changes were made to this version by the publisher prior to publication.The final version of record is available at http://dx.

Extraction and investigation of biosignal features for visual discomfort evaluation

“…The objective measurements require observing the body response during stereoscopic perception of differently rendered stereoscopic images. The most widely used tools to follow the body response in the research field are eye-tracking devices (Bernhard et al 2014;Iatsun et al 2015;Lin, Kao 2018) and brain scanning devices (Fischmeister, Bauer 2006;Frey et al 2016;Moon, Lee 2017). However, other means to measure visual discomfort are also investigated, e.g., Lee et al 2016 showed it is possible to measure visual discomfort using facial expressions.…”

“…Atliekant objektyvų vertinimą yra stebimas savanorių biologinis atsakas į rodomą turinį. Stebint biologinių signalų pokytį dažniausiai yra naudojama akių sekimo įranga (Bernhard et al 2014;Iatsun et al 2015;Lin, Kao 2018), bei smegenų aktyvumo matavimo įranga (Fischmeister, Bauer 2006;Frey et al 2016;Moon, Lee 2017). Žinoma, yra ieškoma ir kitų objektyvių būdų vertinti regos diskomfortą, savo tyrime, Lee et al 2016 parodė, kad regos diskomfortą galima vertinti remiantis veido išraiškomis.…”

Evaluation of preference of multimedia content using deep neural networks for electroencephalography

“…Classification accuracy [7] Gaussian naive Bayes classifier leave-one-video-out for each subject 0.502 (F1-score) [14] Relevance vector machine leave-one-video-out for each subject 0.65 (F1-score) [15] Ensemble classifier* leave-one-trial-out 0.647 [16] Support vector machine leave-one-video-out for each subject 0.705 [17] Deep belief network five-fold cross-validation for each subject 0.867 (F1-score) [18] Recurrent neural network four-fold cross-validation 0.880 *Ensemble of support vector machine, nearest mean, 1-nearest neighbor, k-nearest neighbor, and linear discriminant analysis scaling for content delivery [12] and personalized multimedia recommendation [13]. A limitation of the existing EEG-based implicit QoE assessment systems is that their performance still remains at insufficient levels for real-world applications where high reliability is critical.…”

Study of image quality using event‐related potentials and eye tracking measurement

“…However, the results of NR metrics do not always have an ideal correlation with the perceived image quality. In general, subjective evaluation methods can be categorized into explicit and implicit approaches . Explicit evaluation methods are designed to achieve the subjective perception of participants while viewing the images in the specified experimental environment in the form of scoring.…”

“…In general, subjective evaluation methods can be categorized into explicit and implicit approaches. 13 Explicit evaluation methods are designed to achieve the subjective perception of participants while viewing the images in the specified experimental environment in the form of scoring. The ITU-R and ITU-T Recommendations have developed detailed information to conduct the explicit assessment experiments.…”