LiDeR: Lightweight Dense Residual Network for Video Super-Resolution on Mobile Devices

Çetinkaya, Ekrem; Nguyen, Minh; Timmerer, Christian

doi:10.1109/ivmsp54334.2022.9816346

Cited by 4 publications

(5 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Generally, the framework for deep learning-based VSR algorithms includes one alignment module, one feature extraction and fusion module, and one reconstruction module [36]. FSRCNN [37], SRGAN [38], CARN [39], EDVR [40], LiDeR [41], and RBPN [42] are among well-known image/video super-resolution approaches.…”

Section: B Dnn-based Video Super-resolutionmentioning

confidence: 99%

DeepStream: Video Streaming Enhancements using Compressed Deep Neural Networks

Amirpour

Ghanbari

Timmerer

2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

In HTTP Adaptive Streaming (HAS), each video is divided into smaller segments, and each segment is encoded at multiple pre-defined bitrates to construct a bitrate ladder. To optimize bitrate ladders, per-title encoding approaches encode each segment at various bitrates and resolutions to determine the convex hull. From the convex hull, an optimized bitrate ladder is constructed, resulting in an increased Quality of Experience (QoE) for end-users. With the ever-increasing efficiency of deep learning-based video enhancement approaches, they are more and more employed at the client-side to increase the QoE, specifically when GPU capabilities are available. Therefore, scalable approaches are needed to support end-user devices with both CPU and GPU capabilities (denoted as CPU-only and GPU-available end-users, respectively) as a new dimension of a bitrate ladder. To address this need, we propose DeepStream, a scalable content-aware per-title encoding approach to support both CPU-only and GPU-available end-users. (i) To support backward compatibility, DeepStream constructs a bitrate ladder based on any existing per-title encoding approach. Therefore, the video content will be provided for legacy end-user devices with CPU-only capabilities as a base layer (BL). (ii) For high-end end-user devices with GPU capabilities, an enhancement layer (EL) is added on top of the base layer comprising lightweight video super-resolution deep neural networks (DNNs) for each bitrate-resolution pair of the bitrate ladder. A content-aware video super-resolution approach leads to higher video quality, however, at the cost of bitrate overhead. To reduce the bitrate overhead for streaming content-aware video super-resolution DNNs, DeepCABAC, context-adaptive binary arithmetic coding for DNN compression, is used. Furthermore, the similarity among (i) segments within a scene and (ii) frames within a segment are used to reduce the training costs of DNNs. Experimental results show bitrate savings of 34% and 36% to maintain the same PSNR and VMAF, respectively, for GPU-available end-users, while the CPU-only users get the desired video content as usual.

show abstract

Section: B Dnn-based Video Super-resolutionmentioning

confidence: 99%

DeepStream: Video Streaming Enhancements using Compressed Deep Neural Networks

Amirpour

Ghanbari

Timmerer

2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

show abstract

“…All clients run simultaneously in all scenarios. Each client requests one video where video 1 is streamed to clients (1,11,21,31,41,51), video 2 is streamed to clients (2,12,22,32,42,52), and so on. The time slot duration is set to 52 milliseconds in all experiments.…”

Section: Evaluation Setupmentioning

confidence: 99%

“…60% of the videos' segments are stored in each cache server randomly in most scenarios (only in one scenario, we store 30% to 60% videos segments on cache servers). Moreover, each client requests one video where video 1 is streamed to clients (1,11,21,31,41,51,61,71,81,91), video 2 is streamed to clients (2,12,22,32,42,52,62,72,82,92), and so on. The time slot duration is adjusted to 32 milliseconds in all experiments.…”

Section: Evaluation Setupmentioning

confidence: 99%

“…Finally, the frames upscaled by SR must be combined to produce the reconstructed video. We run the most recent and popular video SR models, namely, ESPCN [134], LiDeR [31], FSRCNN [47], EDSR [99], CARN and CARN-M [3], NINASR [70],…”

Section: Super-resolution Modelsmentioning

confidence: 99%

“…However, mobile battery consumption is still a constraint compared to standard desktop PCs. LiDeR [31] is designed considering the constraints of mobile devices based on a lightweight dense residual neural network to operate on decoded video sequences in real-time without hardware-specific optimization.…”

Section: Super-resolution Modelsmentioning

confidence: 99%

See 2 more Smart Citations