A Survey on Automatic Image Captioning

Srivastava, Gargi; Srivastava, Rajeev

doi:10.1007/978-981-13-0023-3_8

Cited by 13 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Automatic Image Captioning has been the subject of a number of recently published survey papers [11][12][13][14]. In this section, only papers related to combining DL with NLP are cited, as these are the closest to the research described in the present work.…”

Section: Automatic Image Captioningmentioning

confidence: 99%

CAPTION: Caption Analysis with Proposed Terms, Image of Objects, and Natural Language Processing

et al. 2022

View full text Add to dashboard Cite

This paper proposes a novel algorithm, called CAPTION, for identifying and correcting errors in automatically generated image captions. The algorithm combines Deep Learning (DL) for object detection in images with Natural Language Processing techniques. CAPTION has been tested in the following three tasks: (1) classify a caption as correct or not; (2) detect wrong words in the caption, and (3) suggest text corrections. Results show that our method is superior with respect to others evaluated in the same data set in the error correction task. These other methods are generally based exclusively on DL models. This work shows that, although semantics still has not been used at its fullest in this type of task, a combination of DL with Natural Language Processing tools presents a better overall performance than using DL methods alone.

show abstract

Section: Automatic Image Captioningmentioning

confidence: 99%

CAPTION: Caption Analysis with Proposed Terms, Image of Objects, and Natural Language Processing

et al. 2022

View full text Add to dashboard Cite

show abstract

“…In recent years, profound advances have been made in deep learning technology due to the breakthrough in computing power of computers and the surge in data ( LeCun et al, 2015 ). Meanwhile, image caption based on deep learning has also seen significant improvements ( Bai and An, 2018 ; Srivastava and Srivastava, 2018 ; Liu et al, 2019 ). Image caption is the intersection of the fields of computer vision and natural language processing, along with its potential value in terms of contributing to visually impaired individuals’ daily life assistance, graphic conversion, automatic title generation and machine intelligence ( Hossain et al, 2019 ; Kang and Hu, 2022 ).…”

Section: Introductionmentioning

confidence: 99%

An image caption model based on attention mechanism and deep reinforcement learning

Bai,

Zhou,

Pang

et al. 2023

Front. Neurosci.

View full text Add to dashboard Cite

Image caption technology aims to convert visual features of images, extracted by computers, into meaningful semantic information. Therefore, the computers can generate text descriptions that resemble human perception, enabling tasks such as image classification, retrieval, and analysis. In recent years, the performance of image caption has been significantly enhanced with the introduction of encoder-decoder architecture in machine translation and the utilization of deep neural networks. However, several challenges still persist in this domain. Therefore, this paper proposes a novel method to address the issue of visual information loss and non-dynamic adjustment of input images during decoding. We introduce a guided decoding network that establishes a connection between the encoding and decoding parts. Through this connection, encoding information can provide guidance to the decoding process, facilitating automatic adjustment of the decoding information. In addition, Dense Convolutional Network (DenseNet) and Multiple Instance Learning (MIL) are adopted in the image encoder, and Nested Long Short-Term Memory (NLSTM) is utilized as the decoder to enhance the extraction and parsing capability of image information during the encoding and decoding process. In order to further improve the performance of our image caption model, this study incorporates an attention mechanism to focus details and constructs a double-layer decoding structure, which facilitates the enhancement of the model in terms of providing more detailed descriptions and enriched semantic information. Furthermore, the Deep Reinforcement Learning (DRL) method is employed to train the model by directly optimizing the identical set of evaluation indexes, which solves the problem of inconsistent training and evaluation standards. Finally, the model is trained and tested on MS COCO and Flickr 30 k datasets, and the results show that the model has improved compared with commonly used models in the evaluation indicators such as BLEU, METEOR and CIDEr.

show abstract

“…A popular computer vision goal is to create algorithms that can replicate a human's ability to caption any image [9,27,44]. Presently, we are witnessing an exciting transition where this dream of automated captioning is advancing into a reality, with automated image captioning now a feature available in several popular technology services.…”

Section: Introductionmentioning

confidence: 99%

Captioning Images Taken by People Who Are Blind

Gurari¹,

Zhao²,

Zhang³

et al. 2020

Preprint

View full text Add to dashboard Cite

While an important problem in the vision community is to design algorithms that can automatically caption images, few publicly-available datasets for algorithm development directly address the interests of real users. Observing that people who are blind have relied on (human-based) image captioning services to learn about images they take for nearly a decade, we introduce the first image captioning dataset to represent this real use case. This new dataset, which we call VizWiz-Captions, consists of over 39,000 images originating from people who are blind that are each paired with five captions. We analyze this dataset to (1) characterize the typical captions, (2) characterize the diversity of content found in the images, and (3) compare its content to that found in eight popular vision datasets. We also analyze modern image captioning algorithms to identify what makes this new dataset challenging for the vision community. We publicly-share the dataset with captioning challenge instructions at https://vizwiz.org.

show abstract

A Survey on Automatic Image Captioning

Cited by 13 publications

References 21 publications

CAPTION: Caption Analysis with Proposed Terms, Image of Objects, and Natural Language Processing

CAPTION: Caption Analysis with Proposed Terms, Image of Objects, and Natural Language Processing

An image caption model based on attention mechanism and deep reinforcement learning

Captioning Images Taken by People Who Are Blind

Contact Info

Product

Resources

About