In this paper, a highly efficient pixel compensator architecture for the H.264/AVC standard is proposed which can provide both inter and intra prediction functions for luma and chroma components of pixels. By decomposing the algorithms used for both prediction methods into small micro-operation steps, a suitable common arithmetic unit architecture capable for performing these operations has been determined. Next, taking into account the possible reference sample transfer scenario, the overall compensator architecture consists of some buffers and multiple common arithmetic units considering is proposed. Since both arithmetic units and the intermediate data buffer for both inter and intra prediction processes have been shared, our integrated design can achieve more than 31% reduction of gate count compared with the sum of the separate designs. Our design can also lead to more than 37% saving of gate count compared with the previous designs. Our compensator can decode the videos up to HDTV resolution, and be applied for the dedicated H.264 hardware codec for various consumer devices.