High Dynamic Range Image Compression Based on Visual Saliency

Abstract

In recent years, large visual language models (LVLMs) have shown impressive performance and promising generalization capability in multi-modal tasks, thus replacing humans as receivers of visual information in various application scenarios. In this paper, we pioneer to propose a variable bitrate image compression framework consisting of a pre-editing module and an end-to-end codec to achieve promising rate-accuracy performance for different LVLMs. In particular, instead of optimizing an adaptive pre-editing network towards a particular task or several representative tasks, we propose a new optimization strategy tailored for LVLMs, which is designed based on the representation and discrimination capability with token-level distortion and rank. The pre-editing module and the variable bitrate end-to-end image codec are jointly trained by the losses based on semantic tokens of the large model, which introduce enhanced generalization capability for various data and tasks. Experimental results demonstrate that the proposed framework could efficiently achieve much better rate-accuracy performance compared to the state-of-the-art coding standard, Versatile Video Coding. Meanwhile, experiments with multi-modal tasks have revealed the robustness and generalization capability of the proposed framework.

Salient Object Detection: An Accurate and Efficient Method for Complex Shape Objects

“…Compared with the rough version, the fine version shows more detailed information for adding the FFM. We believe our method is superior because of the following two aspects: (1) In the rough version, although we reduce the number of model parameters as much as possible, we obtain better results because we use the double-branch network mode and the edge enhancement module to focus on edge information detection. (2) In the fine version, we give fine feedback based on the rough version; hence, we can obtain better results.…”

“…S ALIENT object detection aims to locate the principal objects in an image or video. It is widely applied as a preprocessing procedure in computer vision tasks, including image compression [1]- [3], image segmentation [4], [5], image recognition [6], [7], image classification [8], [9], image retrieval [10], object tracking [11], scene classification [12], video segmentation [13] and action detection [14]. Most previous works were based on the contrast, such as color contrast [15] and global contrast [16], play the most important role in saliency detection.…”

A Novel Unified Model for Multi-exposure Stereo Coding Based on Low Rank Tucker-ALS and 3D-HEVC

“…The HDR coding methods that support backward compatibility are based on tone mapping models [26], [27]. Huang et al [26] proposed an HDR compression method based on the matting Laplacian.…”

“…The HDR compression scheme formulated as an optimization problem and extract LDR image with enhanced details, mitigating severe edge effects or color artifacts. Li et al [27] encoded tone mapped LDR image into a JPEG compatible base layer codestream. They applied inverse tone mapping operation to approximate the HDR image from the reconstructed LDR image.…”

Multi‐exposure image fusion based on feature evaluation with adaptive factor

“…The digital images taken by mobile phones or digital cameras are low dynamic range (LDR) images, so their dynamic range cannot accurately reflect the real-world dynamic range. High dynamic range (HDR) images taken by professional cameras have a more comprehensive dynamic range and are closer to the human visual system (HVS) [2]. The image sensors of digital cameras, such as the charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS), have a limited dynamic range [3].…”