Super-resolution based bitrate adaptation for HTTP adaptive streaming for mobile devices

Nguyen, Minh Hieu; Çetinkaya, Ekrem; Hellwagner, Hermann; Timmerer, Christian

doi:10.1145/3510450.3517322

Cited by 8 publications

(9 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Consistent with recent mainstream works [14,31,63], this paper focuses on the video streaming performance on the ingest side, i.e., involving the uplink from the streamer to the server. A practical issue is that the streamer's uplink bandwidth is usually far lower than the server's downlink bandwidth [9,38,68]. As a result, the streamer has to compress the video bitstreams in low quality to save the delivery bitrate.…”

Section: Video Streaming Infrastructurementioning

confidence: 99%

“…First, the streamer downscales the original highresolution video frames into the low-resolution version, then encodes the frames into video bitstreams for network transmission. The media server decodes the bitstreams as a series of frames and fed them into a neural super-resolution (SR) model for quality enhancement [22,38,61,63,68]. The final restored video holds comparable visual experience as the original version, thus can be applied to the content distribution in different downstream tasks.…”

Section: Neural-enhanced Video Streamingmentioning

confidence: 99%

“…The final restored video holds comparable visual experience as the original version, thus can be applied to the content distribution in different downstream tasks. Recently, optimizing the NVS pipeline has become a hot topic, including improving video encoding efficiency [9,14], reducing steaming protocol latency [62], designing adaptive bit-rate delivery [9] and optimizing SR enhancement [38,59,67]. Overall, the core objective of an efficient NVS paradigm is to improve the rate-distortion trade-off for video delivery, i.e., reducing the streaming bitrate while restricting the quality drop of restored videos.…”

Section: Neural-enhanced Video Streamingmentioning

confidence: 99%

“…Unfortunately, a practical issue is that the streamer's uplink bandwidth is usually far lower than the server's downstream bandwidth [9,38,68], making the ingest side become the performance bottleneck for continuously delivering high-definition videos [14,31,63]. To fit the limited uplink bandwidth, a natural methodology is to compress the video bitstreams in low quality for realizing communicationefficient delivery, e.g., from 1080P/720P to 360P.…”

Section: Introductionmentioning

confidence: 99%

“…Obviously, compression from either perspective will deprive the server from obtaining high-quality videos and finally hampers the perceptual experience of downstream distribution services, especially when video quality often significantly impacts user's engagement in video streaming [11,13,26,51,71]. To alleviate the performance bottleneck on the ingest side in current video streaming infrastructure, it comes the rise of Neural-enhance Video Steaming (NVS) paradigm [9,14,22,31,61,63], where a majority of works aim to restore the compressed video quality by deploying a neural super-resolution (SR) model on the media server [22,38,55,61,63,68]. The restored videos should hold sufficient visual quality as the original version, so as to guarantee the service experience of downstream video distribution.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

CaDM: Codec-aware Diffusion Modeling for Neural-enhanced Video Streaming

Zhou¹,

Li²,

Guo³

et al. 2022

Preprint

View full text Add to dashboard Cite

Recent years have witnessed the dramatic growth of Internet video traffic, where the video bitstreams are often compressed and delivered in low quality to fit the streamer's uplink bandwidth. To alleviate the quality degradation, it comes the rise of Neural-enhanced Video Streaming (NVS), which shows great prospects to recover low-quality videos by mostly deploying neural super-resolution (SR) on the media server. Despite its benefit, we reveal that current mainstream works with SR enhancement have not achieved the desired rate-distortion trade-off between bitrate saving and quality restoration, due to: (1) overemphasizing the enhancement on the decoder side while omitting the codesign of encoder, (2) inherent limited restoration capacity to generate high-fidelity perceptual details, and (3) optimizing the compression-and-restoration pipeline from the resolution perspective solely, without considering color bitdepth. Aiming at overcoming these limitations, we are the first to conduct the encoder-decoder (i.e., codec) synergy by leveraging the visual-synthesis genius of diffusion models. Specifically, we present the Codec-aware Diffusion Modeling (CaDM), a novel NVS paradigm to significantly reduce streaming delivery bitrate while holding pretty higher restoration capacity over existing methods. First, CaDM improves the encoder's compression efficiency by simultaneously reducing resolution and color bit-depth of video frames. Second, CaDM provides the decoder with perfect quality enhancement by making the denoising diffusion restoration aware of encoder's resolution-color conditions. Evaluation on public cloud services with Open-MMLab benchmarks shows that CaDM significantly saves streaming bitrate by a nearly 100× reduction over vanilla H.264 and achieves much better recovery quality (e.g., FID of 0.61) over state-of-the-art neural-enhancing methods.

show abstract

Section: Video Streaming Infrastructurementioning

confidence: 99%

Section: Neural-enhanced Video Streamingmentioning

confidence: 99%