KENGIC: KEyword-driven and N-Gram Graph based Image Captioning

Abstract

This paper presents a Keyword-driven and N-gram Graph based approach for Image Captioning (KENGIC). Most current state-of-the-art image caption generators are trained endto-end on large scale paired image-caption datasets which are very laborious and expensive to collect. Such models are limited in terms of their explainability and their applicability across different domains. To address these limitations, a simple model based on N-Gram graphs which does not require any end-toend training on paired image captions is proposed. Starting with a set of image keywords considered as nodes, the generator is designed to form a directed graph by connecting these nodes through overlapping n-grams as found in a given text corpus. The model then infers the caption by maximising the most probable n-gram sequences from the constructed graph. To analyse the use and choice of keywords in context of this approach, this study analysed the generation of image captions based on (a) keywords extracted from gold standard captions and (b) from automatically detected keywords. Both quantitative and qualitative analyses demonstrated the effectiveness of KENGIC. The performance achieved is very close to that of current state-of-the-art image caption generators that are trained in the unpaired setting. The analysis of this approach could also shed light on the generation process behind current top performing caption generators trained in the paired setting, and in addition, provide insights on the limitations of the current most widely used evaluation metrics in automatic image captioning.

A Bibliometric Analysis of Publications on Knowledge Risks as a Subfield of Knowledge Management

“…For example, the bigrams in the text "a company at risk" are: {"a company", "company at", "at risk"}. An N-gram graph is a graph which connects N-grams, initially proposed in by Giannakopoulos et al (2008) as a method of summarizing (Birmingham & Muscat, 2022). The meaning that can be deduced from the results offered by these instruments, for the analysis of the thematic evolution of a scientific field, is regarding the thematic concentration.…”