Adaptive in-band motion compensated temporal filtering based on motion mismatch detection in the highpass subbands

Abstract

This paper presents an adaptive in-band motion compensated temporal filtering (MCTF) scheme for 3-D wavelet based scalable video coding. The proposed scheme overcomes the problem of visual artifacts due to motion mismatch when motion information of the spatial lowpass subband is inaccurately applied to the highpass subbands in decoding high spatial resolution video. More specifically, the proposed scheme determines if the mismatch energy is beyond an acceptable threshold in the wavelet domain and then adaptively switches between using the motion information from the spatial lowpass and highpass subbands in the MCTF prediction step. As a result, the amount of motion overhead needs to be transmitted is minimized and a good trade-off between motion information and texture is realized. Experimental results show that the proposed scheme greatly improves the quality of the decoded video at high spatial resolutions while achieving a high coding efficiency. Furthermore, the proposed adaptive scheme requires only modifications when performing MCTF in the highpass subbands, hence the original strength of in-band MCTF for decoding low spatial resolution video is well preserved.

Macroblock-Level Mode Based Adaptive in-Band Motion Compensated Temporal Filtering

“…Furthermore, MBs in areas with smooth motions may in fact be better predicted in terms of reducing the prediction error energy using the MVs of the collocated lowpass subbands' MBs [7]. Table 1 compares the number of motion bits generated by performing ME on the second 64 frames of the foreman sequence using the approaches from [5] [7] [8]. As can be seen, although the subband-adaptive scheme reduces the overall motion bits significantly comparing with the multi-scheme, some highpass subbands' MVs are in fact unnecessarily transmitted to the decoder.…”

“…In [8], we proposed a subband-level adaptive in-band MCTF scheme (denoted the subband-adaptive scheme) that removes the motion mismatch by selectively transmitting the MVs of the entire related highpass spatial subbands. This approach assumes that when mismatch occurs in 1 MB in 1 highpass subband, it is also likely to occur in other MBs in the current and the rest highpass subbands.…”

“…The objective of the proposed approach is therefore to find a more efficient way in the MV selection process to eliminate motion mismatch as well as suppressing the MCTF prediction error, so that both the visual quality and PSNR performances can be improved. Table 1: Number of motion bits generated by a 4-level MCTF of the second 64 frames (2nd GOP for bitstream truncation [11]) for CIF foreman using the multi- [5], single [7] and subband-adaptive [8] in-band schemes, 1-level pre-temporal 9/7 DWT is used…”

“…However, the cross-band motion information correlation is not well exploited since some macroblocks (MBs) in the highpass subbands do not need their own MVs to be transmitted to the decoder. We extend the idea in [8], and propose a MB-level adaptive in-band MCTF scheme that transmits only the necessary highpass subbands' MVs so that a better motion-texture trade-off can be achieved. The MV selection decision is made by detecting motion mismatch on the MB-level in the highpass spatial subbands.…”

“…However, if the MVs from the LL subband do not capture the underlying motion in the highpass subbands, mismatch artefacts will appear in the decoded video. In [8], a subband-based adaptive approach has been proposed. It removes the mismatch by additionally transmitting the highpass subband MVs.…”