No-Reference Video Quality Estimation Based on Machine Learning for Passive Gaming Video Streaming Applications

Barman, Nabajeet; Jammeh, Emmanuel; Ghorashi, Seyed Ali; Martini, Maria G.

doi:10.1109/access.2019.2920477

Cited by 59 publications

(54 citation statements)

References 47 publications

(66 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Also, within ITU-T Study Group 12 another work item, G.OMG was established with the aim of developing a QoEbased gaming model for predicting the overall quality based on the characteristics of the network, system, as well as player and usage context factors. In addition, there are several research works regarding objective and subjective quality assessment of gaming video content such as creation of gaming video datasets [331], evaluation of existing metrics [332]- [335] and development of new no-reference metrics and models [46], [336], [337] for gaming content.…”

Section: B Qoe In Cloud Gaming Video Streaming Applicationsmentioning

confidence: 99%

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

Barakabitze

Barman

Ahmad

et al. 2020

IEEE Commun. Surv. Tutorials

Self Cite

160

View full text Add to dashboard Cite

The highly demanding Over-The-Top (OTT) multimedia applications pose increased challenges to Internet Service Providers (ISPs) for assuring a reasonable Quality of Experience (QoE) to their customers due to lack of flexibility, agility and scalability in traditional networks. The future networks are shifting towards the cloudification of the network resources via Software Defined Networks (SDN) and Network Function Virtualization (NFV). This will equip ISPs with cutting-edge technologies to provide service customization during service delivery and offer QoE which meets customers' needs via intelligent QoE control and management approaches. Towards this end, we provide in this paper a tutorial and a comprehensive survey of QoE management solutions in current and future networks. We start with a highlevel description of QoE management for multimedia services, which integrates QoE modelling, monitoring, and optimization. This followed by a discussion of HTTP Adaptive Streaming (HAS) solutions as the dominant technique for streaming videos over the best-effort Internet. We then summarize the key elements in SDN/NFV along with an overview of ongoing research projects, standardization activities and use cases related to SDN, NFV, and other emerging applications. We provide a survey of the state-of-the-art of QoE management techniques categorized into three different groups: a) QoE-aware/driven strategies using SDN and/or NFV; b) QoE-aware/driven approaches for adaptive streaming over emerging architectures such as multi-access edge computing, cloud/fog computing, and information-centric networking; and c) extended QoE management approaches in new domains such as immersive augmented and virtual reality, mulsemedia and video gaming applications. Based on the review, we present a list of identified future QoE management challenges regarding emerging multimedia applications, network management and orchestration, network slicing and collaborative service management in softwarized networks. Finally, we provide a discussion on future research directions with a focus on emerging research areas in QoE management, such as QoEoriented business models, QoE-based big data strategies, and scalability issues in QoE optimization.

show abstract

Section: B Qoe In Cloud Gaming Video Streaming Applicationsmentioning

confidence: 99%

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

Barakabitze

Barman

Ahmad

et al. 2020

IEEE Commun. Surv. Tutorials

Self Cite

160

View full text Add to dashboard Cite

show abstract

“…Nofu as a NR metric has a slightly higher performance compared to VMAF as FR metric on the Gam-ingVideoSET [6]. Two other NR metrics for gaming content are proposed in [3], named NR-GVSQI and NR-GVSQE. These two proposed models are designed using supervised learning algorithms based on MOS and VMAF values as the target output.…”

Section: Related Workmentioning

confidence: 99%

“…For the validation of each investigated CNN, all frames and VMAF values of the encoded videos of the remaining six source video sequences were selected. In addition, we used all frames of the 144 encoded videos and their VMAF values of the KUGVD dataset [3] for evaluating the model in each step to avoid any bias from the training. This leads to a total of 210.600 frames for the validation.…”

Section: Fundamental Design Phasementioning

confidence: 99%

See 1 more Smart Citation

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

Utke

Zadtootaghaj

Schmidt

et al. 2020

Multimed Tools Appl

View full text Add to dashboard Cite

Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud gaming, e.g. Nvidia GeForce NOW and Google Stadia. In contrast to traditional video content, gaming content has special characteristics such as extremely high and special motion patterns, synthetic content and repetitive content, which poses new opportunities for the design of machine learning-based models to outperform the state-of-the-art video and image quality approaches for this special computer generated content. In this paper, we train a Convolutional Neural Network (CNN) based on an objective quality model, VMAF, as ground truth and fine-tuned it based on subjective image quality ratings. In addition, we propose a new temporal pooling method to predict gaming video quality based on frame-level predictions. Finally, the paper also describes how an appropriate CNN architecture can be chosen and how well the model performs on different contents. Our result shows that among four popular network architectures that we investigated, DenseNet performs best for image quality assessment based on the training dataset. By training the last 57 convolutional layers of DenseNet based on VMAF values, we obtained a high performance model to predict VMAF of distorted frames of video games with a Spearman's Rank correlation (SRCC) of 0.945 and Root Mean Score Error (RMSE) of 7.07 on the image level, while achieving a higher performance on the video level leading to a SRCC of 0.967 and RMSE of 5.47 for the KUGVD dataset. Furthermore, we fine-tuned the model based on subjective quality ratings of images from gaming content which resulted in a SRCC of 0.93 and RMSE of 0.46 using one-hold-out cross validation. Finally, on the video level, using the proposed pooling method, the model achieves a very good performance indicated by a SRCC of 0.968 and RMSE of 0.30 for the used gaming video dataset.

show abstract

“…Furthermore, it highlights the need for a more stable FR-metric tailored to the specific context of GVS. In a third, subsequent study [27], they present two quality predicting models for passive GVS [19].…”

Section: Related Workmentioning

confidence: 99%

A low-complexity psychometric curve-fitting approach for the objective quality assessment of streamed game videos

Damme

Vega

Heyse

et al. 2020

Signal Processing: Image Communication

View full text Add to dashboard Cite

The increasing popularity of video gaming competitions, the so called eSports, has contributed to the rise of a new type of end-user: the passive game video streaming (GVS) user. This user acts as a passive spectator of the gameplay rather than actively interacting with the content. This content, which is streamed over the Internet, can suffer from disturbing network and encoding impairments. Therefore, assessing the user's perceived quality, i.e. the Quality of Experience (QoE), in real-time becomes fundamental. For the case of natural video content, several approaches already exist that tackle the client-side real-time QoE evaluation. The intrinsically different expectations of the passive GVS user, however, call for new real-time quality models for these streaming services. Therefore, this paper presents a real-time Reduced-Reference (RR) quality assessment framework based on a low-complexity psychometric curve-fitting approach. The proposed solution selects the most relevant, low-complexity objective feature. Afterwards, the relationship between this feature and the ground-truth quality is modelled based on the psychometric perception of the human visual system (HVS). This approach is validated on a publicly available dataset of streamed game videos and is benchmarked against both subjective scores and objective models. As a side contribution, a thorough accuracy analysis of existing Objective Video Quality Metrics (OVQMs) applied to passive GVS is provided. Furthermore, this analysis has led to interesting insights on the accuracy of low-complexity client-based metrics as well as to the creation of a new Full-Reference (FR) objective metric for GVS, i.e. the Game Video Streaming Quality Metric (GVSQM).

show abstract

No-Reference Video Quality Estimation Based on Machine Learning for Passive Gaming Video Streaming Applications

Cited by 59 publications

References 47 publications

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

QoE Management of Multimedia Streaming Services in Future Networks: A Tutorial and Survey

NDNetGaming - development of a no-reference deep CNN for gaming video quality prediction

A low-complexity psychometric curve-fitting approach for the objective quality assessment of streamed game videos

Contact Info

Product

Resources

About