Generating Image Captions in Arabic using Root-Word Based Recurrent
            Neural Networks and Deep Neural Networks

Jindal, Vasu

doi:10.18653/v1/n18-4020

Cited by 14 publications

(5 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The approach consists of using deep belief network and root words, in which tree based relations are used to generate captions in the right order. This work was further followed by [16], in which the author replaces the deep belief networks with recurrent neural networks. However, the work proposed in this document uses the multi-task paradigm to overcome the shortage of data in the Arabic language.…”

Section: Related Workmentioning

confidence: 99%

Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigm

Za’ter¹,

Talafha²

2022

Preprint

View full text Add to dashboard Cite

The continuous increase in the use of social media and the visual content on the internet have accelerated the research in computer vision field in general and the image captioning task in specific. The process of generating a caption that best describes an image is a useful task for various applications such as it can be used in image indexing and as a hearing aid for the visually impaired. In recent years, the image captioning task has witnessed remarkable advances regarding both datasets and architectures, and as a result, the captioning quality has reached an astounding performance. However, the majority of these advances especially in datasets are targeted for English, which left other languages such as Arabic lagging behind. Although Arabic language, being spoken by more than 450 million people and being the most growing language on the internet, lacks the fundamental pillars it needs to advance its image captioning research, such as benchmarks or unified datasets. This work is an attempt to expedite the synergy in this task by providing unified datasets and benchmarks, while also exploring methods and techniques that could enhance the performance of Arabic image captioning. The use of multi-task learning is explored, alongside exploring various word representations and different features. The results showed that the use of multi-task learning and pre-trained word embeddings noticeably enhanced the quality of image captioning, however the presented results show that Arabic captioning still lags behind when compared to the English language. The used dataset and code are available at this link.Keywords Image Captioning • Arabic Language • Multi-task Learning • Benchmark However, the overwhelming majority of the available datasets, algorithms, and research efforts were directed toward English, which leaves other languages lagging in this task. Arabic language for instance, is one of the majorly used languages on the web, used by more than 450 million people, and also among internet users, Arabic is the language with the fastest growth rate in the last few years [8]. Despite these facts, the image captions generation was untouched

show abstract

Section: Related Workmentioning

confidence: 99%

Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigm

Za’ter¹,

Talafha²

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, some of the researchers also utilized BiLSTM [14]. Alongside English researchers also generated captions in Chinese [15,16], Japanese [17], Arabic [18] and Bahasa Indonesia [19].…”

Section: Image Captioning In Other Languagesmentioning

confidence: 99%

Bornon: Bengali Image Captioning with Transformer-Based Deep Learning Approach

et al. 2021

View full text Add to dashboard Cite

Image captioning using encoder-decoder-based approach where CNN is used as the Encoder and sequence generator like RNN as Decoder has proven to be very effective. However, this method has a drawback, that is, sequence needs to be processed in order. To overcome this drawback, some researchers have utilized the transformer model to generate captions from images using English datasets. However, none of them generated captions in Bengali using the transformer model. As a result, we utilized three different Bengali datasets to generate Bengali captions from images using the transformer model. Additionally, we compared the performance of the transformer-based model with a visual attention-based encoder-decoder approach. Finally, we compared the result of the transformer-based model with other models that employed different Bengali image captioning datasets.

show abstract

“…For Arabic computer vision, limited related work currently exists. Several authors have worked on the generation of Arabic captions for images (Jindal, 2018;Almuzaini et al, 2018). In another paper, a new dataset related to Arabic computer vision was built.…”

Section: Abdulkareem Alsudaismentioning

confidence: 99%

Extending ImageNet to Arabic using Arabic WordNet

Alsudais¹

2020

Proceedings of the First Workshop on Advances in Language and Vision Research

View full text Add to dashboard Cite

ImageNet has millions of images that are labeled with English WordNet synsets. This paper investigates the extension of ImageNet to Arabic using Arabic WordNet. The objective is to discover if Arabic synsets can be found for synsets used in ImageNet. The primary finding is the identification of Arabic synsets for 1,219 of the 21,841 synsets used in ImageNet, which represents 1.1 million images. By leveraging the parent-child structure of synsets in ImageNet, this dataset is extended to 10,462 synsets (and 7.1 million images) that have an Arabic label, which is either a match or a direct hypernym, and to 17,438 synsets (and 11 million images) when a hypernym of a hypernym is included. When all hypernyms for a node are considered, an Arabic synset is found for all but four synsets. This represents the major contribution of this work: a dataset of images that have Arabic labels for 99.9% of the images in ImageNet.

show abstract

Generating Image Captions in Arabic using Root-Word Based Recurrent Neural Networks and Deep Neural Networks

Cited by 14 publications

References 21 publications

Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigm

Bench-Marking And Improving Arabic Automatic Image Captioning Through The Use Of Multi-Task Learning Paradigm

Bornon: Bengali Image Captioning with Transformer-Based Deep Learning Approach

Extending ImageNet to Arabic using Arabic WordNet

Contact Info

Product

Resources

About