Interactions Guided Generative Adversarial Network for unsupervised image captioning

Cao, Shan; An, Gaoyun; Zheng, Zhenxing; Ruan, Qiuqi

doi:10.1016/j.neucom.2020.08.019

Cited by 34 publications

(18 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This systematic summary approach provides a more concise overview of the development of image captioning technology. Positive results have been achieved with current unsupervised and partially supervised image captioning techniques [92][93][94][95]99] based on the question of whether training data is needed or whether image-text pairs are required. For the different forms of captions, in addition to the mainstream whole picture based caption, there are also image paragraph caption [100][101][102][103][104], and dense caption [8,77,105,106].…”

Section: Discussionmentioning

confidence: 99%

“…(3) From 2019 to now, image captioning is in a boom phase of development. New techniques such as Transformer, reinforcement learning and GANs have been widely applied to solve image description problems, and unsupervised image captioning methods [92][93][94][95] become a new research hotspot. The form of captioning has become more diverse as it is no longer confined to the overall content of the image [58,81,96].…”

Section: Evolutionary Path Of Image Captioningmentioning

confidence: 99%

See 1 more Smart Citation

A Scientometric Visualization Analysis of Image Captioning Research From 2010 to 2020

Liu¹,

et al. 2021

IEEE Access

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Section: Evolutionary Path Of Image Captioningmentioning

confidence: 99%

A Scientometric Visualization Analysis of Image Captioning Research From 2010 to 2020

Liu¹,

et al. 2021

IEEE Access

View full text Add to dashboard Cite

“…Nowadays, deep learning is being widely used in many computer vision applications and it obtains state of the art performance in various domains. Particularly, since Krizhevsky et al [6] achieved the first place on the ImageNet classification and localization challenges in 2012, deep learning has been significantly applied to various problems, such as object detection [7], [8], image captioning [9], microscopic image analysis [10], analysis of skin marks [11], [12] and remote-sensing [13]. Moreover, the pre-trained CNNs can be used to generate high-level feature descriptors to represent visual objects in images.…”

Section: Introductionmentioning

confidence: 99%

Colour Neural Descriptors for Instance Retrieval Using CNN Features and Colour Models

et al. 2021

View full text Add to dashboard Cite

Image representations in the form of neural activations derived from intermediate layers of deep neural networks are the state-of-the-art descriptors for instance based retrieval. However, the problem that persists consists of how to retrieve identical images as the most relevant ones from a large image or video corpus. In this work, we introduce colour neural descriptors that are made of convolutional neural networks (CNN) features obtained by combining different colour spaces and colour channels. In contrast to previous works, which rely on fine-tuning pre-trained networks, we compute the proposed descriptors based on the activations generated from a pretrained VGG-16 network without fine-tuning. Besides, we take advantage of an object detector to optimize our proposed instance retrieval architecture to generate features at both local and global scales. In addition, we introduce a stride based query expansion technique to retrieve objects from multi-view datasets. Finally, we experimentally proved that the proposed colour neural descriptors, obtain state-of-the-art results in Paris 6K, Revisiting-Paris 6k, INSTRE-M and COIL-100 datasets, with mAPs of 81.70, 82.02, 78.8 and 97.9, respectively.

show abstract

“…Image superresolution is the task of transforming the original image into higher resolution and greater detail [7]. Image captioning is also a transformation task, wherein a deep learning algorithm learns to describe the main contents of an image in text media captured as caption [8].…”

Section: Introductionmentioning

confidence: 99%

Translation of Real-World Photographs into Artistic Images via Conditional CycleGAN and StarGAN

Komatsu

Gonsalves

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

To translate a real-world photograph into an artistic image in the style of a famous artist, the selection of colors and brushstrokes should reflect those of the artist. A one-to-one domain translation architecture, CycleGAN, trained with an unpaired dataset can be used to translate a real-world photograph into an artistic image. However, to translate images in N number of multi-artistic styles, the disadvantage is that more than one CycleGAN must be trained corresponding to each style. Here, we develop a single deep learning architecture that can be controlled to yield multiple artistic styles by adding a conditional vector. The overall architecture includes a one-to-N domain translation architecture, namely, a conditional CycleGAN, and an N-to-N domain translation architecture, namely, StarGAN, for translating into five different artistic styles. An evaluation of the trained models reveal that multiple artistic styles can be produced from a single real-world photograph only by adjusting the conditional input.

show abstract

Interactions Guided Generative Adversarial Network for unsupervised image captioning

Cited by 34 publications

References 10 publications

A Scientometric Visualization Analysis of Image Captioning Research From 2010 to 2020

A Scientometric Visualization Analysis of Image Captioning Research From 2010 to 2020

Colour Neural Descriptors for Instance Retrieval Using CNN Features and Colour Models

Translation of Real-World Photographs into Artistic Images via Conditional CycleGAN and StarGAN

Contact Info

Product

Resources

About