Optimal nonlinear sampling for video streaming at low bit rates

Abstract

Over low-bit-rate channels, we adopt the streaming of nonlinearly sampled video frames (i.e., key-frame slideshow) synchronized with the audio stream. Given the channel and buffer limits, we wish to obtain a set of sampled frames that is not only feasible (i.e., streamable), but also optimal in terms of maximal information flow (given that the semantic information contents of each frame can be quantified either automatically or manually). Different application scenarios are considered and modeled in a principle way, for which we propose computationally efficient algorithms for finding the global optimal solution. The contributions of this paper include the novel modeling schemes for channel and buffer limits in the video temporal sampling problem, the analysis and development of the corresponding efficient algorithms for finding the global optimal solution, and the extension and analysis of these algorithms for practical application scenarios. The proposed algorithms have made possible the automated production of the new form of video streaming over low bit rate channels for devices with limited storage capabilities.Index Terms-Dynamic programming, low bit-rate communication, nonlinear temporal sampling, T-C knapsack problem, video streaming, Z-B diagram.