Study of interpolation filters for motion estimation with application in H.264/AVC encoders

Abstract

Image super-resolution plays an important role in a plethora of applications, including video compression and motion estimation. Detecting fractional displacements among frames facilitates the removal of temporal redundancy and improves the video quality by 2-4 dB PSNR [1] [2]. However, the increased complexity of the Fractional Motion Estimation (FME) process adds a significant computational load to the encoder and sets constraints to real-time designs. Timing analysis shows that FME accounts for almost half of the entire motion estimation period, which in turn accounts for 60−90% of the total encoding time depending on the design configuration. FME bases on an interpolation procedure to increase the resolution of any frame region by generating sub-pixels between the original pixels. Modern compression standards specify the exact filter to use in the Motion Compensation module allowing the encoder and the decoder to create and use identical reference frames. In particular, H.264/AVC specifies a 6-tap filter for computing the luma values of half-pixels and a low cost 2-tap filter for computing quarter-pixels. Even though it is common practice for encoder designers to integrate the standard 6-tap filter also in the Estimation module (before Compensation), the fact is that the interpolation technique used for detecting the displacements (not computing their residual) is an open choice following certain performance trade-offs. Aiming at speeding up the Estimation, a process of considerably higher computational demand than the Compensation, this work builds on the potential to implement a lower complexity interpolation technique instead of the H.264 6-tap filter. We integrate in the Estimation module several distinct interpolation techniques not included in the H.264 standard, while keeping the standard H.264/AVC Compensation to measure their impact on the outcome of the prediction engine. Related bibliography includes both ideas to avoid/replace the standard computations, as well as architecturestargeting the efficient implementation of the H.264 6-tap filtering procedure and the support of its increased memory requirements. To this end, we note that H.264 specifies a kernel with coefficients ⟨1,−5,20,20,−5,1⟩ to be multiplied with six consecutive pixels of the frame (either in column or row format). The resulting six products are accumulated and normalized for the generation of a single half-pixel (between 3 rd and 4 th tap). The operation must be repeated for each “horizontal” and “vertical” half-pixelby sliding the kernel on the frame, both in row and column order. Moreover, there exist as many “diagonal” half-pixels to be generated by applying the kernel on previously computed horizontal or vertical half-pixels. That is to say, depending on its position, we must process 6 or 36 frame pixels to compute a single half-pixel. To avoid the costly H.264 filter in the Estimation module, we study similar interpolation techniques using less than 6 taps, possibly exploiting gradients on the image. Section II shows three c...

Single-image super-resolution using low complexity adaptive iterative back-projection

“…In this paper, we give emphasis on lowcomplexity and edge-adaptive techniques, because preserving edge information is crucial for the back-projection of the error [12]. We examine a set of well-known interpolation techniques, as well as our edge-adaptive CxScale and CrossHD algorithms [1]. Each technique is utilized as K and/or L to measure the performance of the algorithm in practice and to select the optimal combination/parameterization.…”

“…We adopt the CxScale kernel to compute halfhorizontal and half-vertical error values when upsampling the error matrix by 2 in each dimension. 6) CrossHD: An improvement over edge-detection via Data-Dependent Triangulation, CrossHD was proposed in [1]. It consists of gradient-based checks adjacent to the required sample.…”

“…On the other hand, the LC configuration describes directional interpolation filters depending on the coordinates of the sample we require to calculate. The half-horizontal and half-vertical samples are computed using the following sixtap filter 3, −17, 78, 78, −17, 3 which could also be inferred as a low-complexity approximation of the third-order Lanczos kernel [1]. The half-diagonal samples are computed using a non-separable 12-tap filter.…”

“…1) Bicubic: This is a well known 4-tap FIR filter [14] that requires up to 16 samples when upsampling by a factor of 4. We examined the parameterized polynomial for a = − 1 as stated in [1] which produces the following coefficients : −1, 5, 5, −1 . It also provided the highest objective quality for the images we used.…”

“…Our work examines the eight-tap half-pixel filter −3, 12, −39, 158, 158, −39, 12, −3 which is described in [20]. 5) CxScale: CxScale was proposed in [1] as an edgedirected interpolation technique. It examines 8 neighboring samples per each input sample, using simple gradient checks.…”