Boosting Lightweight Single Image Super-resolution via Joint-distillation

Luo, Xiaotong; Liang, Q.; Liu, Ding; Qu, Yanyun

doi:10.1145/3474085.3475288

Cited by 6 publications

(3 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In contrast, self-distillation needs no extra network except for the network itself. While self-distillation has been successfully applied in computer vision and natural language processing [50]- [52], it focuses on unimodal tasks.…”

Section: Knowledge Distillationmentioning

confidence: 99%

A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations

Wang

Lin

et al. 2024

IEEE Trans. Multimedia

View full text Add to dashboard Cite

Emotion recognition in conversations (ERC), the task of recognizing the emotion of each utterance in a conversation, is crucial for building empathetic machines. Existing studies focus mainly on capturing context-and speakersensitive dependencies on the textual modality but ignore the significance of multimodal information. Different from emotion recognition in textual conversations, capturing intra-and intermodal interactions between utterances, learning weights between different modalities, and enhancing modal representations play important roles in multimodal ERC. In this paper, we propose a transformer-based model with self-distillation (SDT) 1 for the task. The transformer-based model captures intra-and inter-modal interactions by utilizing intra-and inter-modal transformers, and learns weights between modalities dynamically by designing a hierarchical gated fusion strategy. Furthermore, to learn more expressive modal representations, we treat soft labels of the proposed model as extra training supervision. Specifically, we introduce self-distillation to transfer knowledge of hard and soft labels from the proposed model to each modality. Experiments on IEMOCAP and MELD datasets demonstrate that SDT outperforms previous state-of-the-art baselines.

show abstract

Section: Knowledge Distillationmentioning

confidence: 99%

A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations

Wang

Lin

et al. 2024

IEEE Trans. Multimedia

View full text Add to dashboard Cite

show abstract

“…After SRKD, PISR [16] and JDSR [17] were developed. PISR uses the high-frequency components of the HR image as privileged information to generate a high-performance teacher model, and then transfers knowledge from the teacher model to the student model by measuring the feature difference between the teacher and student models in an embedding space with Gaussian or Laplacian distribution.…”

Section: Knowledge Distillation In Sr Domainmentioning

confidence: 99%

“…Therefore, we have focused on applying KD (which is less dependent on hardware) to SR models. SRKD [15], PISR [16], and JDSR [17] are the examples that apply KD to the SR domain. As an initial study case, SRKD used feature distillation to learn the feature distribution of the teacher model so that the student model resembles it.…”

Section: Introductionmentioning

confidence: 99%

TAKDSR: Teacher Assistant Knowledge Distillation Framework for Graphics Image Super-Resolution

Yoon,

Lee,

Song

2023

IEEE Access

View full text Add to dashboard Cite

This paper presents a framework for effectively applying knowledge distillation (KD) to superresolution (SR) tasks for computer graphics (CG) images. Specifically, we propose TAKDSR, a KD framework for SR using a teacher assistant (TA) network. Recently, the performance of SR models has improved dramatically thanks to the development of deep learning. SR models have evolved into a form that requires a considerable amount of computation and parameters while adopting a complex neural network structure to improve performance. However, it is difficult to utilize conventional high-performance SR models for real-time up-scaling in CG applications requiring high resolution and high frame rate. To solve this, we employ an approach that applies KD to a lightweight SR model. At this time, if the high-resolution (HR) image is used as input for the teacher to show superior performance to the student, a large performance difference occurs between the two due to the excessive performance of the teacher. As a result, the teacher's knowledge has a significantly hard and complex nature, and when transferred to the student, the effect of KD can be rather weakened. Therefore, we adopt a TA network to facilitate the propagation of knowledge between teacher and student. At the same time, the distribution of compact features (CF), which are the decoder input of the teacher, is discretized so that it is compatible with the input distribution of the student, enabling effective KD. Experimental results demonstrate the proposed TAKDSR significantly improves the performance of a given SR model on CG image datasets.

show abstract

A unified architecture for super-resolution and segmentation of remote sensing images based on similarity feature fusion

Wang,

Liu

et al. 2024

Displays

View full text Add to dashboard Cite

Boosting Lightweight Single Image Super-resolution via Joint-distillation

Cited by 6 publications

References 17 publications

A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations

A Transformer-Based Model With Self-Distillation for Multimodal Emotion Recognition in Conversations

TAKDSR: Teacher Assistant Knowledge Distillation Framework for Graphics Image Super-Resolution

A unified architecture for super-resolution and segmentation of remote sensing images based on similarity feature fusion

Contact Info

Product

Resources

About