H.264-Based Compression of Bayer Pattern Video Sequences

Abstract

Most consumer digital cameras use a single light sensor which captures color information using a color filter array (CFA). This produces a mosaic image, where each pixel location contains a sample of only one of three colors, either red, green or blue. The two missing colors at each pixel location must be interpolated from the surrounding samples in a process called demosaicking. The conventional approach to compressing video captured with these devices is to first perform demosaicking and then compress the resulting full-color video using standard methods. In this paper two methods for compressing CFA video prior to demosaicking are proposed. In our first method, the CFA video is directly compressed with the H.264 video coding standard in 4:2:2 sampling mode. Our second method uses a modified version of H.264, where motion compensation is altered to take advantage of the properties of CFA data. Simulations show both proposed methods give better compression efficiency than the demosaick-first approach at high bit rates, and thus are suitable for applications, such as digital camcorders, where high quality video is required. Index Terms-Bayer pattern, color demosaicking, H264/AVC, single-sensor digital video cameras, video coding. I. INTRODUCTION M OST commercial digital cameras use a single light sensor which is monochromatic in nature. In order to capture RGB color information, a color filter array (CFA) is used, which produces a mosaic image where each pixel location contains either a red, green or blue sample. The Bayer pattern CFA [1] is commonly used, which captures pixels in groups of four; each group containing two green, one red and one blue sample (Fig. 1). More green samples are captured than red or blue because the human visual system is more sensitive to the green portion of the light spectrum. Other CFA patterns are possible [2] but the Bayer pattern is considered here due to its commercial importance. In order to form a full color image or video from CFA data, the color data is interpolated in a process called demosaicking Manuscript

H.264/AVC based color filter array compression with inter-channel prediction model

“…The demosaicked G pixels are used just for the prediction, and they need not be transmitted to the decoder. Experimental results show that the first approach brings the compression gain of 0.74dB, 0.53dB, and 0.55dB for each channel over the existing CFA compression method in [5], and the second gives 1.45dB, 1.07dB and 1.06dB gain.…”

“…This method has been further refined in [3] and a CFA video compression method was also proposed in [4]. Also, the structure conversion in the RGB space was considered for the direct use of H.264/AVC with RGB 4:2:2 color space [5], where a reference picture demosaicking method has also been proposed to avoid the aliasing problem when applying the compression-first approach to the motion picture compression. Doutre and Nasiopoulos [6] introduced a new green channel intra prediction method for H.264/AVC which considers actual pixel location of green channel at the original CFA image.…”

“…In the conventional H.264/AVC based CFA compression methods [5,6], the data shown in Fig. 1 is first restructured to Fig.…”

“…In the compression-first approach, the CFA image is restructured to be an RGB 4:2:2 image based on the structure conversion method [5] without G channel demosaicking. Let us denote the origianl CFA image in Fig.…”

“…Then it is compressed by the standard image compression methods. This conventional approach is called "demosaick-first" in contrast to the recently developed "compression-first" approaches that have been proved to be more efficient than the demosaick-first [1][2][3][4][5][6]. The main reason for the inferiority of former approach is that some redundant information is generated while demosaicking.…”