English to Tamil Multi-Modal Image Captioning Translation

Kumar, V H Vishnu; Lalithamani, N.

doi:10.1109/aic55036.2022.9848810

Cited by 3 publications

(4 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Bilingual Evaluation Understudy Score (BLEU), an evaluation tool, is used in the research to examine multi-modal systems. This research [93] aims to bridge the gap between technology and end users by developing a system that produces image descriptions in Tamil using multiple architectures. This could enhance internet search results and assist visually impaired individuals.…”

Section: Image Captioning For Tamil Languagementioning

confidence: 99%

A Comprehensive Survey on Image Captioning for Indian Languages: Techniques, Datasets, and Challenges

Jayaswal,

Rani,

Kaur

2023

Preprint

View full text Add to dashboard Cite

In image captioning, we generate visual descriptions from an image. Image Cap-tioning requires identifying the key entity, feature, and association in an image. There is also a requirement to generate captions that are syntactically and semantically correct. The process of image captioning requires computer vision and natural language processing. In the past few decades, a substantial attempt has been made to generate the caption for images. In this survey article, we are going to present an extensive survey on image captioning for Indian Languages. To summarize recent research work in image captioning, first, we briefly review the traditional approach to image captioning depending on template and retrieval. Further deep-learning approaches for image captioning are concentrated which are classified as encoder-decoder architecture, attention-based approach, and transformer architecture. Our main focus in this survey is based on image cap-tioning techniques for Indian languages like Hindi, Bengali Assamese, etc. After that, we analyze the state-of-the-art approach on the most widely dataset i.e. MS COCO dataset with their strengths, limitations, and performance metrics i.e. BLEU, ROUGE, METEOR, CIDEr, SPICE. At last, we explore discussion on open challenges and future direction in the field of image captioning.

show abstract

Section: Image Captioning For Tamil Languagementioning

confidence: 99%

A Comprehensive Survey on Image Captioning for Indian Languages: Techniques, Datasets, and Challenges

Jayaswal,

Rani,

Kaur

2023

Preprint

View full text Add to dashboard Cite

show abstract

“…The employed model adopts a fused architecture within an encoder-decoder framework, using Inception-V3 CNN to serve as a visual feature extractor from images and a long short-term memory (LSTM) unit as a sequential caption generator. (Vishnu et al, 2022 [17]) employed a multimodal transformer-based architecture to create text captions in the Tamil language. They developed Tamil MSCOCO dataset by translating the captions into Tamil.…”

Section: Image Captioning In Tamil and Telugu Languagesmentioning

confidence: 99%

“…In one of the Tamil captioning works by (Rajalingam et al, 2021 [16]), the authors employed machine translation techniques to translate the generated English captions to Tamil, on the MSCOCO dataset . The other two works, one for Tamil Captioning by (Vishnu et al, 2022 [17]) and the other for Telugu captioning by (Dwarampudi et al, 2021 [15]), utilized the CNN-LSTM model on the Flickr8k dataset. To assess the performance of our models, we conducted a comparison of the BLEU scores with the aforementioned models, and the results are in Table 10.…”

Section: Qualitative Analysismentioning

confidence: 99%

“…The development of image captioning models in Tamil and Telugu languages has been rather limited, primarily relying on initial methodologies such as the CNN-RNN architecture (Dwarampudi et al ., 2021 [15], Rajalingam et al, 2021 [16]) and the caption translation technique (Vishnu et al, 2022 [17]). While these approaches have shown promising progress, they have not been able to produce truly creative captions that fully grasp the intricate details of the images.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Multilingual Image Captioning: Multimodal Framework for Bridging Visual and Linguistic Realms in Tamil and Telugu through Transformers

Gunti,

et al. 2023

Preprint

View full text Add to dashboard Cite

Image captioning is a formidable challenge in the realms of computer vision and natural language processing. The limited availability of captioning mechanisms for non-English languages poses significant barriers for nonproficient English speakers. Specifically, Tamil and Telugu the most spoken languages in India, lack image captioning models capable of delivering accurate captions. Moreover, generating captions in Tamil and Telugu is a complex task due to their unique linguistic intricacies. Addressing these challenges requires advanced models capable of capturing long-range dependencies and generating contextually meaningful image captions. This research presents a multimodal deep learning framework that appropriately integrates InceptionV3, VGG16, and ResNet50 convolutional neural network architectures with multihead attention-based transformer architecture. By harnessing the multihead attention mechanism, our model effectively comprehends image context, handles linguistic complexity, and establishes vital multimodal associations between visual and textual features. Extensive experiments were carried out on translated versions of the benchmark datasets such as Flickr8k, Flickr30k, and MSCOCO to evaluate the efficacy of the model. The multimodal technique we suggested produced extraordinary and remarkable results especially in terms of BLEU metrics. The maximum value of the BLEU-1 score achieved by our suggested model reaches 65.16 and 66.79 on Tamil and Telugu caption generation tasks respectively. These findings outperformed the results of existing methods, indicating improved accuracy in generating captions for both Tamil and Telugu languages. Furthermore, a meticulous, manually labored audit of the generated captions confirmed their appropriateness and competence, affirming the robustness of the proposed methodology.

show abstract

Modelling Complex Associations for Image Captioning Using Vision Transformers

Ritul,

Sampath

2024

2024 IEEE 9th International Conference for Convergence in Technology (I2CT)

View full text Add to dashboard Cite

English to Tamil Multi-Modal Image Captioning Translation

Cited by 3 publications

References 0 publications

A Comprehensive Survey on Image Captioning for Indian Languages: Techniques, Datasets, and Challenges

A Comprehensive Survey on Image Captioning for Indian Languages: Techniques, Datasets, and Challenges

Multilingual Image Captioning: Multimodal Framework for Bridging Visual and Linguistic Realms in Tamil and Telugu through Transformers

Modelling Complex Associations for Image Captioning Using Vision Transformers

Contact Info

Product

Resources

About