Wyner-Ziv side information generation using a higher order piecewise trajectory temporal interpolation algorithm

Abstract

Abstract-Distributed video coding (DVC) reverses the traditional coding paradigm of complex encoders allied with basic decoding, to one where the computational cost is largely incurred by the decoder. This enables low-cost, resource-poor sensors to be used at the transmitter in various applications including multi-sensor surveillance. A key constraint governing DVC performance is the quality of side information (SI), a coarse representation of original video frames which are not available at the decoder. Techniques to generate SI have generally been based on linear temporal interpolation, though these do not always produce satisfactory SI quality especially in sequences exhibiting asymmetric (non-linear) motion. This paper presents a higherorder piecewise trajectory temporal interpolation (HOPTTI) algorithm for SI generation that quantitatively and perceptually affords better SI quality in comparison to existing temporal interpolation-based approaches.

Improving distributed video coding side information by intelligently combining macro-blocks from multiple algorithms

“…Though HOPTTI improved SI generation [10] due to more accurate motion modeling, the MV estimation using BMA algorithm can sometimes have issues of MB overlapping caused by inaccuracies in forward and backward trajectories or blocky artifacts caused by multiple or deformable objects in a single MB. AOBMC has been employed with HOPTTI [11] (known as HOPTTI-AOBMC) to tackle the above mentioned issues by allowing pixels of the surrounding blocks to moderate the predicted MV, as illustrated in Figure 1, then, using a raised cosine weighting window enclosing the neighboring MBs and the size of the window is determined by the reliability of the MV of neighboring MBs and their distance from the MB under consideration.…”

“…AOBMC has been employed with HOPTTI [11] (known as HOPTTI-AOBMC) to tackle the above mentioned issues by allowing pixels of the surrounding blocks to moderate the predicted MV, as illustrated in Figure 1, then, using a raised cosine weighting window enclosing the neighboring MBs and the size of the window is determined by the reliability of the MV of neighboring MBs and their distance from the MB under consideration. This is achieved by minimizing SBAD [10], which measures the spatial error of the MB under consideration between the reference and current HOPTTI frames. Since the original current frame is not available at the decoder, the previous key frame is used as the reference frame [7].…”

“…The trajectory is built using the MVs of MBs in the previous and future key frames and using motion compensated interpolation to predict intermediate object positions and the SI frames [10]. To calculate the four parameters ji, ai, vi and di, a minimum of 4 key frames are required, and if it is assumed the respective displacements of the blocks at these key frames are Ai, Bi, Ci and Di, then the following holds:…”

“…SI is commonly generated using linear-motion compensated temporal interpolation (LMCTI) [3][4][5] and while this generally provides reasonable quality, it does not always afford an adequate formulation as motion in real sequences is not always linear. More accurate SI have been generated in DVC by the modelling of higher-order trajectories [8][9][10][11] which includes the HOPTTI algorithm [10]. As the higher order trajectory is determined by using three or more motion vectors (MV) from previous and future frames, HOPTTI is able to model nonlinear motion more accurately and deliver improvement in SI quality [10].…”

“…More accurate SI have been generated in DVC by the modelling of higher-order trajectories [8][9][10][11] which includes the HOPTTI algorithm [10]. As the higher order trajectory is determined by using three or more motion vectors (MV) from previous and future frames, HOPTTI is able to model nonlinear motion more accurately and deliver improvement in SI quality [10]. However, due to its use of block matching algorithms (BMA), blocking artefacts and overlapping can be observed especially where abrupt changes occur in motion trajectory and multiple objects occupy the same MB.…”