A novel feature map compression method based on feature transformation for VCM

Abstract

In this paper, we propose a novel feature map compression method for Video Coding for Machines (VCM). The proposed method performs a principal component analysis (PCA)-based transform on feature pyramid network (FPN) feature maps using predefined basis and mean vectors. In addition, the proposed method reduces redundancy between different resolution levels within FPN feature maps based on redundancy between FPN layers. The fixed predefined basis and mean are employed through PCA with a set of training data set. For any input videos, transform coefficients are obtained by performing transform with the fixed basis and compressed using Versatile Video Coding (VVC). Experimental results show that the proposed method achieves 89.22% and 86.57% BD-rate gain compared to the VCM feature anchor in instance segmentation, and object detection, respectively.

Transform-Based Feature Map Compression Method for Video Coding for Machines (VCM)

“…Figure 2 showcases the overall pipeline of the proposed method. Similar to previous works [11][12][13][14], our method utilizes PCA to reduce the dimensions of the multi-level feature maps intended for compression. Previous methods pose certain drawbacks: they necessitate performing an independent PCA for one or more feature maps for each input datum during the encoding process, and they compress the basis In an effort to circumvent these issues, a transform-based feature map compression method employing a predefined generalized basis matrix and mean vector is proposed.…”

“…However, prior studies have indicated that the impact on performance from errors in the feature map domain due to quantization is not substantial [11]. Therefore, the previous feature map compression methods and the MPEG-VCM [22] feature anchor generation process adopted a uniform integer quantization method, relying only on the minimum and maximum values and bit depth [11][12][13][14].…”

“…Table 6 shows BD-rate reductions achieved by our method against the previous PCAbased feature map compression methods [12][13][14] in object detection and instance segmentation for the OpenImage V6 dataset. In the object detection task, the proposed method achieved 85.35% BD-rate reduction against the feature anchor.…”

“…While superior to discrete cosine transform-based compression methods, this approach is limited when compressing low-level feature maps, which constitute more than half the data among multilevel feature maps. In addition, various PCA-based multi-level feature map compression methods have been proposed, such as by Park et al [12], Lee et al [13], and Rosewarne and Nguyen [14]. These methods commonly employ a dimension-reduction mechanism to eliminate redundancy within each feature map and between low-level feature maps, leveraging the properties of the FPN structure.…”

“…These components have distinct characteristics and may necessitate different codecs for sufficient compression efficiency. Depending on spatial redundancy, Park et al [12] and Lee et al [13] utilized VVC and DeepCABAC [15], while Rosewarne and Nguyen [14] allocated these three components into different areas of a picture, defined as a single sub-picture. The sub-picture was compressed by the VVC quantizer at a lower quantization parameter (QP) than the QP for other components.…”