GiGAN: Gate in GAN, could gate mechanism filter the features in image-to-image translation?

Nie, Xuan; Jia, Jianchao; Ding, Haoxuan; Wong, Edward K.

doi:10.1016/j.neucom.2021.07.085

Cited by 3 publications

(5 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Essentially, these methods aim to learn an approximation of a transformation over the query image, which makes the underlying issue an image-to-image translation problem. There are many image translation methods using GANs [8,29,41,58] or more powerful diffusion models [11,56] that can be either used off-the-shelf or updated to work well with our scenario. Although these methods do not necessarily update the actual downstream task model, they still require per-distribution updates for the model, which makes it expensive.…”

Section: Related Workmentioning

confidence: 99%

“…GANs CycleGAN [58] 52.9 U-GAT-IT-light [8] 105.0 NICE-GAN [8] 67.6 N2D-GAN [29] 43.6 SPN2D-GAN [29] 94.0 STGAN [41] 519.0 Table 5: Computational complexity of our method compared to other input-level image enhancement strategies.…”

Section: Modelmentioning

confidence: 99%

“…For instance, autoencoders are used to encode compact representations of input samples and then decode them to minimize a reconstruction loss with respect to ground truth samples [1,32,66]. Generative and/or adversarial models (e.g., diffusion models [11,56,63] or GANs [8,29,40,41,58]) modify input samples by iterative processes or style transfer techniques. The main drawbacks of these methods are the need for expensive inference time and for paired clean-corrupted samples at training time while being able to handle in-domain corruptions only (i.e., particular corruption types seen at training time).…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Modular System for Enhanced Robustness of Multimedia Understanding Networks via Deep Parametric Estimation

Barbato,

Michieli,

Yucel

et al. 2024

Proceedings of the ACM Multimedia Systems Conference 2024 on ZZZ

View full text Add to dashboard Cite

In multimedia understanding tasks, corrupted samples pose a critical challenge, because when fed to machine learning models they lead to performance degradation. In the past, three groups of approaches have been proposed to handle noisy data: i) enhancer and denoiser modules to improve the quality of the noisy data, ii) data augmentation approaches, and iii) domain adaptation strategies. All the aforementioned approaches come with drawbacks that limit their applicability; the first has high computational costs and requires pairs of clean-corrupted data for training, while the others only allow deployment of the same task/network they were trained on (i.e., when upstream and downstream task/network are the same). In this paper, we propose SyMPIE to solve these shortcomings. To this end, we design a small, modular, and efficient (just 2GFLOPs to process a Full HD image) system to enhance input data for robust downstream multimedia understanding with minimal computational cost. Our SyMPIE is pre-trained on an upstream task/network that should not match the downstream ones and does not need paired clean-corrupted samples. Our key insight is that most input corruptions found in real-world tasks can be modeled through global operations on color channels of images or spatial filters with small kernels. We validate our approach on multiple datasets and tasks, such as image classification (on ImageNetC, ImageNetC-Bar, VizWiz, and a newly proposed mixed corruption benchmark named ImageNetC-mixed) and semantic segmentation (on Cityscapes, ACDC, and DarkZurich) with consistent improvements of about 5% relative accuracy gain across the board. 1 The code of our approach and the new ImageNetC-mixed benchmark will be made available upon publication.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Modelmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A Modular System for Enhanced Robustness of Multimedia Understanding Networks via Deep Parametric Estimation

Barbato,

Michieli,

Yucel

et al. 2024

Proceedings of the ACM Multimedia Systems Conference 2024 on ZZZ

View full text Add to dashboard Cite

show abstract

“… between the topic information vector and the hidden state vector for each word can be obtained in equation (4).…”

Section: Integration Of Thematic and Lexical Informationmentioning

confidence: 99%

“…Emotional analysis can help machine translation systems better understand the emotional meaning of the original text. This can accurately convey and preserve the emotional information contained in the original text, avoiding emotional loss or misinformation caused by language and cultural differences, and improving translation quality [3][4]. However, traditional analysis methods mainly focus on the emotional feature extraction and the classifier combination selection.…”

Section: Introductionmentioning

confidence: 99%

Intelligent Text Sentiment Analysis of Emotional Beauty in the English Translation: Based on TB-LSTM Algorithm

2024

JAIT

View full text Add to dashboard Cite

In the existing deep learning sentiment analysis methods, text is directly processed into a sequence of words, which often ignores the important components of the text and the topic of the text. To solve these problems, the topic of text is extracted through Latent Dirichlet Allocation model, and then a method for integrating topic information into neural networks is proposed. Furthermore, an emotion classification model that integrates the topic information is constructed. An attention mechanism is used at the lexical level and the sentence level, paying attention to the more and less important content separately when building the document presentation. The results show that the proposed model has excellent performance in all four kinds of data, among which the accuracy rate, recall rate and F1-score of CCF-BDC are the best, reaching 0.63, 0.72 and 0.7 respectively. The translation accuracy also reaches 0.85. These results show that the model proposed in this study has better performance in sentiment analysis. These research results will help improve the user's reading experience and make machine translation better meet the needs of human language expression.

show abstract

A survey on multimodal-guided visual content synthesis

Zhang

Pan

et al. 2022

Neurocomputing

View full text Add to dashboard Cite

GiGAN: Gate in GAN, could gate mechanism filter the features in image-to-image translation?

Cited by 3 publications

References 26 publications

A Modular System for Enhanced Robustness of Multimedia Understanding Networks via Deep Parametric Estimation

A Modular System for Enhanced Robustness of Multimedia Understanding Networks via Deep Parametric Estimation

Intelligent Text Sentiment Analysis of Emotional Beauty in the English Translation: Based on TB-LSTM Algorithm

A survey on multimodal-guided visual content synthesis

Contact Info

Product

Resources

About