Supporting zoomable video streams with dynamic region-of-interest cropping

Abstract

Streaming of an arbitrary region of interest (RoI) from a high resolution video is essential to supporting zooming and panning within a video stream. This paper explores two methods for RoI-based streaming, referring to them as tiled streaming and monolithic streaming. Tiled streaming partitions video frames into grid of tiles and encodes each tile as an independently decodable stream. Monolithic streaming applies to video encoded using off-the-shelf encoder, and relies on a pre-computed dependency information to send the necessary bits for the RoI. We evaluated these two methods in terms of bandwidth efficiency, storage requirement, and computational costs under different video encoding parameters. Experimental results show that bandwidth efficiency of tiled streams for RoI-based streaming reduces when tile size increases, despite improvement in compression efficiency. In the case of monolithic streams, use of a larger motion vector range coupled with careful run-time optimization can still improve the bandwidth efficiency, despite an increase in motion vector dependency.

Interactive UHDTV at the Commonwealth Games

“…Such a solution has the drawback that the servers need to maintain a processing session and a delivery channel for each individual client. With ROI-based coding [10], a client decoder only needs to access a subset of a compressed video, i.e. the macroblocks containing the relevant pixels, and to track dependencies on other macroblocks/video regions created by the encoder.…”

“…Some of the first comparisons between regular encoding and coding for tiled streaming were investigated by [10]. In particular, they compared regular monolithic streaming with tiled streaming.…”