Adaptive reshaping for next generation video codec

Lu, Taoran; Pu, Fangjun; Yin, Peng; Chen, Tao; Husak, Walt

doi:10.1117/12.2320807

Cited by 2 publications

(1 citation statement)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The offset values which are calculated based on pixels' statistics are sent to the decoder. VVC keeps both DBF and SAO filters, however, adds three more filters: Luma Mapping Chroma Scaling (LMCS) [35], Adaptive Loop Filter (ALF) [7] and, Cross-Component Adaptive Loop Filter (CC-ALF) [36]. LMCS is implemented to improve coding efficiency through two processes: luma mapping (LM) and chroma scaling (CS).…”

Section: Related Work a Traditional In-loop Filtersmentioning

confidence: 99%

Joint Pixel and Frequency Feature Learning and Fusion via Channel-Wise Transformer for High-Efficiency Learned In-Loop Filter in VVC

Kathariya,

Li,

Auwera

2024

IEEE Trans. Circuits Syst. Video Technol.

View full text Add to dashboard Cite

Block-based video codecs such as Versatile Video Coding (VVC)/H.266, High Efficiency Video Coding (HEVC)/H.265, Advanced Video Coding (AVC)/H.264 etc. inherently introduces compression artifacts. Although these codecs have in-loop filters to correct these distortions, they are not always effective due to the complexity of the noise. Recently, deep-learning approaches emerged as a promising solution for in-loop filtering. However, most of the previous approaches were designed solely for learning from images and neglected the high-frequency signals present in the reconstructed video frames. Furthermore, some previous methods employed a multilevel feature-extraction and feature-fusion strategy to enhance performance. However, they utilized complex feature-extractors while relying on naive feature-fusion methods. In this article, we propose a novel framework called TSF-Net, which jointly learns from both the pixel (spatial) and frequency-decomposed information and through powerful capability of a channel-wise transformer, it fuses both these information to improve performance. Our approach deviates from previous approaches by employing a simple feature-extractor coupled with an advanced transformer-based feature-fusion module. Simultaneously, TSF-Net introduces a few fundamental modifications in the multihead self-attention module of the channel-wise transformer to make it computationally efficient. Our experimental results show that the proposed TSF-Net achieves a Bjøntegaard Delta (BD) -bitrate saving of up to 10.258% for the luma (Y) component under all-intra (AI) profile outperforming the VVC baseline and other state-of-the-art methods. Moreover, the proposed TSF-Net with an efficient channel-wise transformer is twice as efficient as TSF-Net with a vanilla channel-wise transformer.

show abstract

Section: Related Work a Traditional In-loop Filtersmentioning

confidence: 99%