Parallelizing video transcoding using Map-Reduce-based cloud computing

Abstract

Due to the complexity of video coding, fast transcoding is still a challenge. Various parallel coding methods have been proposed. In this paper, we present a parallel transcoding system over Map/Reduce cloud computing architecture. Input video sequences are divided into segments, and mapped to multiple computers. The sub-tasks are launched in parallel with processing results concatenated to the final output sequences. For heterogeneous clips, computing capacity, and task-launching overhead, the task scheduling over cloud is an NP-hard problem. We propose a low-complexity heuristic algorithm, Max-MCT, to find out the optimal solutions for task scheduling. By estimating the low-bound of finish time, we transform the problem into a virtual knapsack problem. But it is not an optimal solution for the original problem therefore we use a minimal complete time (MCT) algorithm to minimize the entire finish time. We carry out extensive experiments on numerical simulations. The results verified that our algorithm outperforms the existing algorithms.

An openstack-based flexible video transcoding framework in live

“…The difference is the fact that we do not limit the length of segments. Here, all segments are mapped to multiple computers by MAX scheduling strategy, and then pushed to the cloud [12].…”

Transcoding live adaptive video streams at a massive scale in the cloud

“…Most of these works [7,9,[11][12][13] take advantage of the fact that some modern video compression techniques divide the video stream into non-overlapping Group of Pictures (GOPs) that can be treated independently of each other. The encoding time of each GOP depends on its duration and on the complexity of the corresponding scene.…”

“…Some authors [7,11] propose fairly complicated systems to estimate the encoding time of each GOP based on real-time measurements, while others [9,12,13] assume that this information is directly available, for instance [9] by profiling the encoding of a few representative videos of different types, similarly to what we have done (see section 3.2). Another downside of a GOP-based solution is that the encoding of each GOP can be completed out of order and then need to be reordered before being delivered to the users.…”

“…Lao et al [12] and Lin et al [13] deal only with batches of videos to transcode and present different scheduling algorithms to minimize the overall encoding time.…”

“…For instance, some papers [7,9,[11][12][13]] present algorithms to schedule transcoding jobs on a group of computers, typically in order to maximize CPU utilization [7,11] or to minimize the finishing time [12,13]. Other researchers have analyzed the performance of video encoding and the relationship between power, rate and distortion [8,21,25] using analytical models and empirical studies.…”