Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Gutiérrez, Jesús; Pérez, Pablo; Orduna, Marta; Singla, Ashutosh; Cortés, Carlos; Mazumdar, Pramit; Viola, Irene; Brunnström, Kjell; Battisti, Federica; Cieplińska, Natalia; Juszka, Dawid; Janowski, Lucjan; Leszczuk, Mikołaj; Adeyemi-Ejeye, Anthony Olufemi; Hu, Yaosi; Chen, Zhengzhong; Wallendael, Glenn Van; Lambert, Peter; Díaz, César; Hedlund, John; Hamsis, Omar; Fremerey, Stephan; Hofmeyer, Frank; Raake, Alexander; César, Pablo; Carli, Marco; García, Narciso

doi:10.1109/tmm.2021.3093717

Cited by 44 publications

(29 citation statements)

References 80 publications

(99 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Quality. The aggregate quality was asked, following the literature [8], in the post-sequence questionnaire using the Absolute Category Rating (ACR) on a fve-grade quality scale [21], where the categories: "Bad", "Poor", "Fair", "Good", and "Excellent" were presented.…”

Section: Methodsmentioning

confidence: 99%

“…On the one hand, the omnidirectional cameras that ofer better qualities imply high costs. On the other hand, stable network conditions are required to transmit 360-degree video with acceptable quality in real time [8,17]. In this sense, several works in the literature have analyzed efcient coding schemes and the industry is working hard to bring the technology closer to a greater part of society [12,34].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of the Performance of an Immersive System for Tele-education

Orduna

Gutiérrez

Sánchez

et al. 2022

ACM International Conference on Interactive Media Experiences

Self Cite

View full text Add to dashboard Cite

Tele-education was already a solution for people who cannot attend lessons in person (such as inaccessibility in rural areas or illness issues). However, COVID has revealed problems in tele-education with current technology, causing adolescents and children to slow down their learning curves and experience problems of social distancing with their classmates. This paper presents a user study to validate an immersive communication system for tele-education purposes. This system streams in real time a class using 360-degree cameras, allowing remote students to explore the whole scene and improving the feeling of being in the classroom with their colleagues. Additionally, the prototype provides notifcations to the remote students about events (such as a changes in the teacher's presentation or classmates raising their hands) that occur outside their viewport to indicate in which direction they should move their heads to visualize them.To validate the system and investigate its possible added value, socioemotional factors such as presence, perceived quality, usability, and usefulness of the notifcations were evaluated through a user test using questionnaires. The obtained results show that using immersive tele-education systems can improve the presence, as

show abstract

Section: Methodsmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Evaluation of the Performance of an Immersive System for Tele-education

Orduna

Gutiérrez

Sánchez

et al. 2022

ACM International Conference on Interactive Media Experiences

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, the acceptable bitrate was found to be content-dependent as more motion required higher bitrates, indicating that motion activity should be further investigated. Recently, Gutierrez, et al (2021) [9] published a cross lab investigation based on the work of the Video Quality Experts Group (VQEG), that has greatly influenced the recent Recommendation from the ITU [17]. The study in [9] was conducted time wise in parallel with this study and studied audio visual quality, simulator sickness symptoms, and exploration behaviour in short 360-video sequences.…”

Section: Related Workmentioning

confidence: 99%

“…Recently, Gutierrez, et al (2021) [9] published a cross lab investigation based on the work of the Video Quality Experts Group (VQEG), that has greatly influenced the recent Recommendation from the ITU [17]. The study in [9] was conducted time wise in parallel with this study and studied audio visual quality, simulator sickness symptoms, and exploration behaviour in short 360-video sequences. It was targeted to study methodological questions on how to conduct subjective experiment with 360-video.…”

Section: Related Workmentioning

confidence: 99%

Quality of experience of 360 video – subjective and eye-tracking assessment of encoding and freezing distortions

Kasteren

Brunnström

Hedlund

et al. 2022

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

The research domain on the Quality of Experience (QoE) of 2D video streaming has been well established. However, a new video format is emerging and gaining popularity and availability: VR 360-degree video. The processing and transmission of 360-degree videos brings along new challenges such as large bandwidth requirements and the occurrence of different distortions. The viewing experience is also substantially different from 2D video, it offers more interactive freedom on the viewing angle but can also be more demanding and cause cybersickness. The first goal of this article is to complement earlier research by Tran, et al. (2017) [39] testing the effects of quality degradation, freezing, and content on the QoE of 360-videos. The second goal is to test the contribution of visual attention as an influence factor in the QoE assessment. Data was gathered through subjective tests where participants watched degraded versions of 360-videos through a Head-Mounted Display with integrated eye-tracking sensors. After each video they answered questions regarding their quality perception, experience, perceptual load, and cybersickness. Our results showed that the participants rated the overall QoE rather low, and the ratings decreased with added degradations and freezing events. Cyber sickness was found not to be an issue. The effects of the manipulations on visual attention were minimal. Attention was mainly directed by content, but also by surprising elements. The addition of eye-tracking metrics did not further explain individual differences in subjective ratings. Nevertheless, it was found that looking at moving objects increased the negative effect of freezing events and made participants less sensitive to quality distortions. More research is needed to conclude whether visual attention is an influence factor on the QoE in 360-video.

show abstract

“…The conventional solution for this problem is conducting targeted experiments to only assess low-level features with shorter stimuli. For instance, to evaluate the visual quality of 360° (Gutierrez et al, 2021) or point clouds (Viola et al, 2022), or the effect of delay in task performance (Brunnström et al, 2020). As an alternative approach, Orduna et al (2022) propose a methodology to combine frequent assessment of low-level features during the execution of the test with the evaluation of high-level features using postexperience questionnaires.…”

Section: Evaluation Of Low-level Featuresmentioning

confidence: 99%

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practises for QoE Assessment

Pérez,

Gonzalez-Sosa,

Gutiérrez

et al. 2022

Preprint

View full text Add to dashboard Cite

Several technological and scientific advances have been achieved recently in the fields of immersive systems (e.g., 360-degree/multiview video systems, augmented/mixed/virtual reality systems, immersive audio-haptic systems, etc.), which are offering new possibilities to applications and services in different communication domains, such as entertainment, virtual conferencing, working meetings, social relations, healthcare, and industry. Users of these immersive technologies can explore and experience the stimuli in a more interactive and personalized way than previous technologies (e.g., 2D video). Thus, considering the new technological challenges related to these systems and the new perceptual dimensions and interaction behaviors involved, a deep understanding of the users' Quality of Experience (QoE) is required to satisfy their demands and expectations. In this sense, it is essential to foster the research on evaluating the QoE of immersive communication systems, since this will provide useful outcomes to optimize them and to identify the factors that can deteriorate the user experience. With this aim, subjective tests are usually performed following standard methodologies (e.g., ITU recommendations), which are designed for specific technologies and services. Although numerous user studies have been already published, there are no recommendations or standards that define common testing methodologies to be applied to evaluate immersive communication systems, such as those developed for images and video. Taking this into account, a revision of the QoE evaluation methods designed for previous technologies is required to develop robust and reliable methodologies for immersive communication systems. Thus, the objective of this paper is to provide an overview of existing immersive communication systems and related user studies, which can help on the definition of basic guidelines and testing methodologies to be used when performing user tests of immersive communication systems, such as 360-degree video-based telepresence, avatar-based social VR, cooperative AR, etc.

show abstract

Subjective Evaluation of Visual Quality and Simulator Sickness of Short 360$^\circ$ Videos: ITU-T Rec. P.919

Cited by 44 publications

References 80 publications

Evaluation of the Performance of an Immersive System for Tele-education

Evaluation of the Performance of an Immersive System for Tele-education

Quality of experience of 360 video – subjective and eye-tracking assessment of encoding and freezing distortions

Emerging Immersive Communication Systems: Overview, Taxonomy, and Good Practises for QoE Assessment

Contact Info

Product

Resources

About