Super-resolution video enhancment based on a constrained set of motion vectors

Abstract

Modern video surveillance and target tracking applications utilize multiple cameras transmitting low-bit-rate video through channels of very limited bandwidth. The highly compressed video exhibits coding artifacts that can cause target detection and tracking procedures to fail. Thus, to lower the level of noise and retain the sharpness of the video frames, super-resolution techniques can be employed for video enhancement. In this paper, we propose an efficient super-resolution video enhancement scheme that is based on a constrained set of motion vectors. The proposed scheme computes the motion vectors using the original (uncompressed) video frames, and transmits only a small set of these vectors to the receiver. At the receiver, each pixel is assigned a motion vector from the constrained set to maximize the motion prediction performance. The size of the transmitted vector set is constrained to be less than 3% of the total coded bit stream. In the video enhancement process, an L2-norm minimization super-resolution procedure is applied. The proposed scheme is applied to enhance highly compressed, real-world video sequences. The results obtained show significant improvement in the visual quality of the video sequences, as well as in the performance of subsequent target detection and tracking procedures.

Super-resolution-based enhancement for real-time ultra-low-bit-rate video coding

“…This SR enhancement scheme showed a significant improvement in both visual quality and PSNR performance of the video. However, as presented in [3,4], the scheme is very computationally complex and not suitable for software-only, real-time video coding applications using general-purpose computing platforms. In our proposed system, we modify the aforementioned SR video enhancement scheme to achieve real-time performance.…”

“…The proposed system adopts the super-resolution (SR) video enhancement scheme proposed in [3,4]. In [3,4], a limited set of motion vectors (LSMV) is selected from the estimated motion vectors based upon their significance in the SR enhancement.…”

“…In [3,4], a limited set of motion vectors (LSMV) is selected from the estimated motion vectors based upon their significance in the SR enhancement. In addition to minimizing the overhead of the motion vectors, artificial motion is added to each frame prior to video compression to increase the SR effectiveness.…”

“…The motion estimation algorithms used in the proposed system are simplified such that they can operate in real time on a general-purpose computing platform. We also incorporate a super-resolution-based video enhancement algorithm, which was developed in [3,4]. The super-resolution enhancement does not require additional information if motion-predicted video coding is already applied.…”

“…The super-resolution enhancement does not require additional information if motion-predicted video coding is already applied. The enhancement algorithm of [3,4] is modified to incorporate a selective enhancement scheme based upon a local difference of reference frames to minimize the effect of erroneous motion prediction. This paper is organized as follows.…”