Effective content-based video caching with cache-friendly encoding and media-aware chunking

Abstract

Caching similar videos transparently in a network is a cost-effective solution that potentially reduces redundant data transfers. Recent study shows that network redundancy elimination (NRE) on the content level could produce high bandwidth savings in ISPs. However, we find that blindly employing existing NRE techniques to video contents could lead to suboptimal redundancy suppression rates. This is because (a) randomness in the video encoding process could produce completely different binaries even when they deal with seemingly identical video clips and (b) existing NRE chunking schemes incur high overheads since they do not utilize the underlying video format.In this work, we present two novel schemes that help similar or aliased videos to be cached more effectively in the NRE system. First, we propose a deterministic video encoding scheme that preserves the unmodified original content even after editing or reencoding. This would eliminate the sources of encoding randomness, allowing the NRE systems to detect the redundancy across similar videos. Second, we propose a lightweight video chunking scheme that exploits the underlying video structure. Our "samplebased" chunking scheme groups the logically-related frames into a chunk, and significantly reduces the size of NRE chunk indexes as well as chunking overheads. Our preliminary evaluation shows that the deterministic video encoding scheme helps greatly expose the redundancy across similar videos even after editing. Also, our sample-based chunking reduces the chunking overhead by a factor of 2.0 to 22.5 compared with popular NRE chunking schemes and reduces the index size by 27 times over various video contents.

Dependency-aware distributed video transcoding in the cloud

“…Based on this approach, we propose a distributed transcoding scheme that segments a video into variable-size chunks according to prediction dependencies mined during the encoding process. The idea of grouping related frames was first suggested in [16] to eliminate network redundancy in video caching and to avoid caching the same video multiple times. The authors defined "sample-based" chunking as grouping all video samples between two consecutive IDR frames.…”