An efficient content-adaptive motion-compensated 3-D DWT with enhanced spatial and temporal scalability

Abstract

We propose a novel, content adaptive method for motion-compensated three-dimensional wavelet transformation (MC 3-D DWT) of video. The proposed method overcomes problems of ghosting and nonaligned aliasing artifacts which can arise in regions of motion model failure, when the video is reconstructed at reduced temporal or spatial resolutions. Previous MC 3-D DWT structures either take the form of MC temporal DWT followed by a spatial transform ("t+2D"), or perform the spatial transform first ("2D + t"), limiting the spatial frequencies which can be jointly compensated in the temporal transform, and hence limiting the compression efficiency. When the motion model fails, the "t + 2D" structure causes nonaligned aliasing artifacts in reduced spatial resolution sequences. Essentially, the proposed transform continuously adapts itself between the "t + 2D" and "2D + t" structures, based on information available within the compressed bit stream. Ghosting artifacts may also appear in reduced frame-rate sequences due to temporal low-pass filtering along invalid motion trajectories. To avoid the ghosting artifacts, we continuously select between different low-pass temporal filters, based on the estimated accuracy of the motion model. Experimental results indicate that the proposed adaptive transform preserves high compression efficiency while substantially improving the quality of reduced spatial and temporal resolution sequences.

Motion-compensated temporal filtering with optimized motion estimation

“…It is observed that in most of the test videos, more than 80% of blocks remain stationary. In a region where there is no motion, the TF results in zero energy in high frequency frame [3].…”

“…It was soon realized that, in order to fully exploit inter-frame redundancy, the temporal part of the transform must compensate for motion between frames that is referred as motion-compensated temporal filtering (MCTF). Without motion compensation a 3D DWT is a separable transform and the order of spatial and temporal DWT stages can be interchanged without altering the final subband [3]. However, this commutative property is lost, with the introduction of motion compensation, as the DWT is not shift invariant.…”

Wavelet based distributed video coding with spatial scalability

“…This improvement attributes to that the full-resolution reconstruction frames comprise higher frequency information. By using the information from higher frequency spatial-subbands, the type II leakage exists in reducedresolution layer can be compensated [2].…”

“…However, the DWT is not a shift invariant operator. Compared to RDWT, spatial domain motion compensation is usually able to yield the better prediction efficiency for side information [2].…”

Macroblock-Level Mode Based Adaptive in-Band Motion Compensated Temporal Filtering

“…Lifting based MCTF performs wavelet transform in two sequential steps, the prediction and the update steps. In our experiments, the bidirectional 5/3 wavelet is used due to its better complexityefficiency trade-off comparing to other wavelet transforms [9]. The prediction and update steps for 5/3 lifting are:…”