Image Captioning using Hybrid LSTM-RNN with Deep Features

Deorukhkar, Kalpana; Ket, Satish

doi:10.1007/s11220-022-00400-7

Cited by 9 publications

(1 citation statement)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The objective is to linguistically generate a possible sentence that is semantically correct to the image content [3]. Thus, language processing and visual understanding of image description are the two major aspects of image captioning [4]. The CV and NLP approaches must be used to properly integrate them and to handle the problem created by the corresponding modalities to guarantee that the generated sentence is semantically and grammatically true.…”

Section: Introductionmentioning

confidence: 99%

Automated Image Captioning Using Sparrow Search Algorithm With Improved Deep Learning Model

Arasi,

Alshahrani,

Alruwais

et al. 2023

IEEE Access

View full text Add to dashboard Cite

Image captioning is a deep learning technique that intends to create and generate textual descriptions or captions for images. It integrates computer vision and natural language processing (NLP) to comprehend the visual content of an image and generate human-like descriptions. Deep learning (DL) based image captioning models can be trained on largescale datasets, allowing them to generalize various types of images and generate captions that apply to a wide range of visual scenarios. By combining computer vision and natural language processing, DL-enabled image captioning models can understand both visual and textual information, which enables them to generate captions that not only describe the visual content but also incorporate contextual and semantic information. This study develops an Automated Image Captioning using Sparrow Search Algorithm with Improved Deep Learning (AIC-SSAIDL) technique. The major intention of the AIC-SSAIDL technique lies in the automated generation of textual captions for the input images. To accomplish this, the AIC-SSAIDL technique utilizes the MobileNetv2 model to generate feature descriptors of the input images and its hyperparameter tuning process takes place using SSA. For the image captioning process, the AIC-SSAIDL technique utilizes an attention mechanism with long short-term memory (AM-LSTM) network. Finally, the hyperparameter selection of the AM-LSTM model is performed by the fruit fly optimization (FFO) algorithm. A wide range of experiments has been conducted on benchmark data to depict the better performance of the AIC-SSAIDL method. The comprehensive result analysis highlighted the enhanced captioning results of the AIC-SSAIDL method with maximum CIDEr of 46.12, 61.89, and 137.45 on Flickr8k, Flickr30k, and MSCOCO datasets, respectively. INDEX TERMS Image captioning; Deep learning; Natural language processing; Sparrow search algorithm; Computer vision.

show abstract

Section: Introductionmentioning

confidence: 99%