Bilateral frame rate upconversion algorithm based on image texture complexity compensation

Abstract

A new bilateral frame rate upconversion algorithm using the image texture complexity characteristic is proposed. In the proposed algorithm, as the cost function is adjusted according to the image texture complexity, the accuracy of motion estimation (ME) is improved. In the experimental results, the proposed algorithm outperforms the conventional bilateral ME process by 0.32 dB.Introduction: Frame rate upconversion (FRUC) is an important technology supporting higher temporal resolution. Recently, FRUC has been used to prevent the motion blur problem in liquid crystal displays [1]. In the previous research, motion-compensated frame interpolation (MCFI) has been shown to be an effective scheme for reducing motion judder [2]. Most MCFI algorithms have used the block-matching algorithm (BMA) for motion estimation (ME) because it is simple and easy to implement [3]. Among the several MCFI methods using BMA, bilateral ME (BME) can avoid holes and overlap problems [4]. In conventional BME, a motion vector (MV) is defined based on the temporal symmetry between previous and following frames. However, the BME has a critical weakness when an object with complex texture and a background with homogeneous texture are simultaneously present. The problem is caused by the simple cost function of conventional BME. To solve this problem, adaptive BME that considers the texture of a block was proposed [5]. However, by just considering the texture of the interpolated image, the cost function of this algorithm does not correctly represent the complexity of the original image, which decreases the accuracy of the ME process. In our proposed methods, the texture complexities of the previous and following frames are used simultaneously, considering that a bilateral cost function increases the accuracy of the ME process. It improves the subjective and objective image quality in the interpolated frame. Considering the substantial quality improvement, the resulting increase in computational complexity is negligible.