Task-Adaptive Attention for Image Captioning

Yan, Chenggang; Hao, Yiming; Li, Liang; Yin, Jian; Liu, An-An; Mao, Zhendong; Chen, Zhenyu; Gao, Xingyu

doi:10.1109/tcsvt.2021.3067449

Cited by 175 publications

(38 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among the seven features, three characteristics are below the standard deviation and two characteristics are between the standard deviation and twice the standard deviation. This value can be obtained by formulas (12) to (16) with similar results for nearly 4 features. Like the statistical results of Fig.…”

Section: Resultssupporting

confidence: 62%

“…Next, we use formulas ( 12) and ( 13) to calculate the feature quantity threshold T σ that takes into account human visual errors. After obtaining the characteristic quantity valve value T σ , calculate the valve value T according to formulas (14) to (16). In the formula, Mo(•) represents the mode, Q 1 (•) is the first interquartile difference, and w k is the weight of the feature.…”

Section: Trademark Similarity Comparison System Based On Visual Weightmentioning

confidence: 99%

“…Therefore, they also proposed some three-dimensional image processing work [14,15] to assist in trademark-related search processing. In addition, Yan et al also discussed the research on the use of natural language in trademarks [16]. Abadi et al [17] used artificial intelligence to classify trademarks registered by the US Trademark Office.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Trademark infringement recognition assistance system based on human visual Gestalt psychology and trademark design

Hung

Chen²,

Chen³

2021

J Image Video Proc.

View full text Add to dashboard Cite

Trademarks are common graphic signs in human society. People used this kind of graphic sign to distinguish the signs of representative significance such as individuals, organizations, countries, and groups. Under effective use, these graphic signs can bring maintenance and development resources and profits to the owner. In addition to maintenance and development, organizations that have obtained resources can further promote national and social progress. However, the benefits of these resources have also attracted the attention of unfair competitors. By imitating counterfeit trademarks that appear, unfair competitors can steal the resources of the original trademark. In order to prevent such acts of unfair competitors, the state has formulated laws to protect trademarks. In the past, there have also been researches on similar trademark searches to assist in trademark protection. Although the original trademark is protected by national laws, unfair competitors have recently used psychological methods to counterfeit the original trademark and steal its resources. Trademarks counterfeited through psychology have the characteristics of confuse consumers and do not constitute infringement under the law. Under the influence of such counterfeit trademarks, the original trademark is still not well protected. In order to effectively prevent such trademark counterfeiting through psychology, this article proposes new features based on trademark design and Gestalt psychology to assist legal judgments. These features correspond to a part of the process that is not fully understood in the human visual system and quantify them. In the experimental results, we used past cases to analyze the proposed assistance system. Discussions based on past judgments proved that the quantitative results of the proposed system are similar to the plaintiff or the judgment to determine the reasons for plagiarism. This result shows that the assistance system proposed in this article can provide visually effective quantitative data, assist the law to prevent malicious plagiarism on images by unfair competitors, and reduce the plagiarism caused by the similar design concepts of late trademark designers.

show abstract

Section: Resultssupporting

confidence: 62%

Section: Trademark Similarity Comparison System Based On Visual Weightmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Trademark infringement recognition assistance system based on human visual Gestalt psychology and trademark design

Hung

Chen²,

Chen³

2021

J Image Video Proc.

View full text Add to dashboard Cite

show abstract

“…Different from conventional image (Liu et al, 2020Li et al, 2020;Yan et al, 2019Yan et al, , 2020aYan et al, , 2021 or video captioning (Deng et al, 2021;Tu et al, 2017Tu et al, , 2020Yan et al, 2020b), change captioning addresses two-image captioning, especially to describe their difference. Jhamtani et al (Jhamtani and Berg-Kirkpatrick, 2018) is the first work for change captioning.…”

Section: Related Workmentioning

confidence: 99%

Semantic Relation-aware Difference Representation Learning for Change Captioning

Tu¹,

Yao²,

Li³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

Change captioning is to describe the difference in a pair of images with a natural language sentence. In this task, the distractors, such as the illumination or viewpoint change, bring the huge challenges about learning the difference representation. In this paper, we propose a semantic relation-aware difference representation learning network to explicitly learn the difference representation in the existence of distractors. Specifically, we introduce a selfsemantic relation embedding block to explore the underlying changed objects and design a cross-semantic relation measuring block to localize the real change and learn the discriminative difference representation. Besides, relying on the POS of words, we devise an attentionbased visual switch to dynamically use visual information for caption generation. Extensive experiments show that our method achieves the state-of-the-art performances on CLEVR-Change and Spot-the-Diff datasets 1 .

show abstract

“…Recently, deep learning [ 5 – 11 ] models have attained significant advancements in the field of medical image analysis by training on enough labeled data and fine-tuning its millions of parameters [ 12 , 13 ]. Therefore, it is becoming more and more important to use deep learning models to analyze CXR images of COVID-19 infected patients, to relieve the shortage of medical resources and the overload of doctors.…”

Section: Introductionmentioning

confidence: 99%

Images denoising for COVID-19 chest X-ray based on multi-resolution parallel residual CNN

Jiang

Zhu

Zheng

et al. 2021

Machine Vision and Applications

View full text Add to dashboard Cite

Chest X-ray (CXR) is a medical imaging technology that is common and economical to use in clinical. Recently, coronavirus (COVID-19) has spread worldwide, and the second wave is rebounding strongly now with the coming winter that has a detrimental effect on the global economy and health. To make pre-diagnosis of COVID-19 as soon as possible, and reduce the work pressure of medical staff, making use of deep learning networks to detect positive CXR images of infected patients is a critical step. However, there are complex edge structures and rich texture details in the CXR images susceptible to noise that can interfere with the diagnosis of the machines and the doctors. Therefore, in this paper, we proposed a novel multi-resolution parallel residual CNN (named MPR-CNN) for CXR images denoising and special application for COVID-19 which can improve the image quality. The core of MPR-CNN consists of several essential modules. (a) Multi-resolution parallel convolution streams are utilized for extracting more reliable spatial and semantic information in multi-scale features. (b) Efficient channel and spatial attention can let the network focus more on texture details in CXR images with fewer parameters. (c) The adaptive multi-resolution feature fusion method based on attention is utilized to improve the expression of the network. On the whole, MPR-CNN can simultaneously retain spatial information in the shallow layers with high resolution and semantic information in the deep layers with low resolution. Comprehensive experiments demonstrate that our MPR-CNN can better retain the texture structure details in CXR images. Additionally, extensive experiments show that our MPR-CNN has a positive impact on CXR images classification and detection of COVID-19 cases from denoised CXR images.

show abstract

Task-Adaptive Attention for Image Captioning

Cited by 175 publications

References 32 publications

Trademark infringement recognition assistance system based on human visual Gestalt psychology and trademark design

Trademark infringement recognition assistance system based on human visual Gestalt psychology and trademark design

Semantic Relation-aware Difference Representation Learning for Change Captioning

Images denoising for COVID-19 chest X-ray based on multi-resolution parallel residual CNN

Contact Info

Product

Resources

About