Rate-Distortion Optimized Motion-Compensated Prediction for Packet Loss Resilient Video Coding

Abstract

A rate-distortion optimized motion-compensated prediction method for robust video coding is proposed. Contrasting methods from the conventional literature, the proposed approach uses the expected reconstructed distortion after transmission, instead of the displaced frame difference in motion estimation. Initially, the end-to-end reconstructed distortion is estimated through arecursive per-pixel estimation algorithm. Then the total bit rate for motion-compensated encoding is predicted using a suitable rate distortion model. The results are fed into the Lagrangian optimization at the encoder to perform motion estimation. Here, the encoder automatically finds an optimized motion compensated prediction by estimating the best tradeoff between coding efficiency and end-to-end distortion. Finally, rate-distortion optimization is applied again to estimate the macroblock mode. This process uses previously selected optimized motion vectors and their corresponding reference frames. It also considers intraprediction. Extensive computer simulations in lossy channel environments were conducted to assess the performance of the proposed method. Selected results for both single and multiple reference frames settings are described. A comparative evaluation using other conventional techniques from the literature was also conducted. Furthermore, the effects of mismatches between the actual channel packet loss rate and the one assumed at the encoder side have been evaluated and reported in this paper.

Optimizing Motion Compensated Prediction for Error Resilient Video Coding

“…The rationale is that by investigating the two extremes of the accuracy-complexity tradeoff, namely, exhaustive encoding versus simple quadratic RD modeling, we effectively bound the performance range. Clearly, any advanced and particularly more accurate RD models (e.g., see [20]) is expected to yield performance somewhere within the "benchmarked range" we provide.…”

“…The current paper subsumes our early work [21], which was the first to propose a basic approach to high accuracy ED-RDO based on extensions to the recursive optimal per-pixel estimate (ROPE) method of [4]. Similar schemes employing ROPE were later proposed in [19] and [20] (the latter also included a higher accuracy rate model).…”

“…Effective solutions to extend ROPE to mitigate mismatch issues are provided in [37] (and references therein), but are not incorporated here so as to avoid diluting the focus. In contrast, [17] uses heuristic distortion estimation while [20] applies ROPE in a practical setting of sub-pixel prediction and in-loop filtering but without mitigating the modeling mismatches, both of which significantly compromise the distortion estimation accuracy. We re-emphasize that our objective here is to bound the range of complexity-precision tradeoff by providing the two extremes of accurate RD modeling via exhaustive encoding, versus low complexity quadratic RD modeling.…”

“…We re-emphasize that our objective here is to bound the range of complexity-precision tradeoff by providing the two extremes of accurate RD modeling via exhaustive encoding, versus low complexity quadratic RD modeling. The approach in [20] uses an advanced heuristic rate model, which is expected to yield performance somewhere within our benchmarked performance range.…”

“…To reduce complexity, a common practice is to e,ploy source coding RD models to efficiently predict and in (17) [20]. Many RD models have been proposed, e.g., [42]- [46].…”