GamingVideoSET: A Dataset for Gaming Video Streaming Applications

Barman, Nabajeet; Zadtootaghaj, Saman; Schmidt, Steven; Martini, Maria G.; Möller, Sebastian

doi:10.1109/netgames.2018.8463362

Cited by 48 publications

(64 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, within ITU-T Study Group 12 another work item, G.OMG was established with the aim of developing a QoEbased gaming model for predicting the overall quality based on the characteristics of the network, system, as well as player and usage context factors. In addition, there are several research works regarding objective and subjective quality assessment of gaming video content such as creation of gaming video datasets [331], evaluation of existing metrics [332]- [335] and development of new no-reference metrics and models [46], [336], [337] for gaming content.…”

Section: B Qoe In Cloud Gaming Video Streaming Applicationsmentioning

confidence: 99%

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

Barakabitze

Barman

Ahmad

et al. 2020

IEEE Commun. Surv. Tutorials

Self Cite

142

View full text Add to dashboard Cite

The highly demanding Over-The-Top (OTT) multimedia applications pose increased challenges to Internet Service Providers (ISPs) for assuring a reasonable Quality of Experience (QoE) to their customers due to lack of flexibility, agility and scalability in traditional networks. The future networks are shifting towards the cloudification of the network resources via Software Defined Networks (SDN) and Network Function Virtualization (NFV). This will equip ISPs with cutting-edge technologies to provide service customization during service delivery and offer QoE which meets customers' needs via intelligent QoE control and management approaches. Towards this end, we provide in this paper a tutorial and a comprehensive survey of QoE management solutions in current and future networks. We start with a highlevel description of QoE management for multimedia services, which integrates QoE modelling, monitoring, and optimization. This followed by a discussion of HTTP Adaptive Streaming (HAS) solutions as the dominant technique for streaming videos over the best-effort Internet. We then summarize the key elements in SDN/NFV along with an overview of ongoing research projects, standardization activities and use cases related to SDN, NFV, and other emerging applications. We provide a survey of the state-of-the-art of QoE management techniques categorized into three different groups: a) QoE-aware/driven strategies using SDN and/or NFV; b) QoE-aware/driven approaches for adaptive streaming over emerging architectures such as multi-access edge computing, cloud/fog computing, and information-centric networking; and c) extended QoE management approaches in new domains such as immersive augmented and virtual reality, mulsemedia and video gaming applications. Based on the review, we present a list of identified future QoE management challenges regarding emerging multimedia applications, network management and orchestration, network slicing and collaborative service management in softwarized networks. Finally, we provide a discussion on future research directions with a focus on emerging research areas in QoE management, such as QoEoriented business models, QoE-based big data strategies, and scalability issues in QoE optimization.

show abstract

Section: B Qoe In Cloud Gaming Video Streaming Applicationsmentioning

confidence: 99%

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

Barakabitze

Barman

Ahmad

et al. 2020

IEEE Commun. Surv. Tutorials

Self Cite

142

View full text Add to dashboard Cite

show abstract

“…In this work, we use GamingVideoSET [19], an open source dataset of gaming video sequences containing subjective and objective quality assessment ratings. The dataset consists of twenty-four uncompressed raw gaming video sequences from twelve different games, each with a 30 second duration, 1080p resolution, and 30 fps frame rate.…”

Section: Datasetmentioning

confidence: 99%

Estimation of Quality Scores From Subjective Tests-Beyond Subjects’ MOS

Pezzulli

Martini

Barman

2021

IEEE Trans. Multimedia

Self Cite

View full text Add to dashboard Cite

Subjective tests for the assessment of the quality of experience (QoE) are typically run with a pool of subjects providing their opinion scores using a 5-point scale. The subjects' mean opinion score (MOS) is generally assumed as the best estimation of the average score in the target population. Indeed, for a large enough sample, we may assume that the mean of the variations across the subjects approaches zero, but this is not the case for the limited number of subjects typically considered in subjective tests. In this paper, we propose an approach based on generalized linear models (GLMs) for estimation of the population average QoE. The motivating dataset is composed of the individual scores assigned by 25 subjects to a set of gaming videos evaluated under different resolutions and compression ratios. The approach recognizes the multinomial nature of the data and allows for correlation between scores of the same subject. The resulting estimated average QoE is shown to follow more credible patterns than the MOS, particularly for higher bitrates, for which the model estimates present more coherent behavior. Similar convincing results are found on a second dataset, showing the validity of the approach.

show abstract

“…Nofu is also a machine learning metric that extracts low level features of which some are hand-crafted by the authors and some are taken from the state of the art. Nofu as a NR metric has a slightly higher performance compared to VMAF as FR metric on the Gam-ingVideoSET [6]. Two other NR metrics for gaming content are proposed in [3], named NR-GVSQI and NR-GVSQE.…”

Section: Related Workmentioning

confidence: 99%

“…GamingVideoSET The GamingVideoSET [6], which we refer to as GVSET in the remainder of the paper, consists of 24 source video sequences from 12 different games (two sequences per game), as shown in Fig. 1…”

Section: Gaming Video Datasetsmentioning

confidence: 99%

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Utke

Zadtootaghaj

Schmidt

et al. 2020

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud gaming, e.g. Nvidia GeForce NOW and Google Stadia. In contrast to traditional video content, gaming content has special characteristics such as extremely high and special motion patterns, synthetic content and repetitive content, which poses new opportunities for the design of machine learning-based models to outperform the state-of-the-art video and image quality approaches for this special computer generated content. In this paper, we train a Convolutional Neural Network (CNN) based on an objective quality model, VMAF, as ground truth and fine-tuned it based on subjective image quality ratings. In addition, we propose a new temporal pooling method to predict gaming video quality based on frame-level predictions. Finally, the paper also describes how an appropriate CNN architecture can be chosen and how well the model performs on different contents. Our result shows that among four popular network architectures that we investigated, DenseNet performs best for image quality assessment based on the training dataset. By training the last 57 convolutional layers of DenseNet based on VMAF values, we obtained a high performance model to predict VMAF of distorted frames of video games with a Spearman's Rank correlation (SRCC) of 0.945 and Root Mean Score Error (RMSE) of 7.07 on the image level, while achieving a higher performance on the video level leading to a SRCC of 0.967 and RMSE of 5.47 for the KUGVD dataset. Furthermore, we fine-tuned the model based on subjective quality ratings of images from gaming content which resulted in a SRCC of 0.93 and RMSE of 0.46 using one-hold-out cross validation. Finally, on the video level, using the proposed pooling method, the model achieves a very good performance indicated by a SRCC of 0.968 and RMSE of 0.30 for the used gaming video dataset.

show abstract

GamingVideoSET: A Dataset for Gaming Video Streaming Applications

Cited by 48 publications

References 7 publications

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

Estimation of Quality Scores From Subjective Tests-Beyond Subjects’ MOS

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Contact Info

Product

Resources

About