LCEval: Learned Composite Metric for Caption Evaluation

Sharif, Naeha; White, Lyndon; Bennamoun, Mohammed; Liu, Wei; Shah, Syed Afaq Ali

doi:10.1007/s11263-019-01206-z

Cited by 10 publications

(5 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SubICap-1k also achieves the highest SPICE and METEOR scores amongst the SOTA models. This shows that our model generates captions which are semantically better than those generated by other models [28].…”

Section: Quantitative Analysismentioning

confidence: 68%

“…7.1, the n-gram based measures tend to overlook the semantics and only focus on the lexical properties of the captions. CIDEr, which is an ngram based measure, [28] prefers captions which have a higher lexical correspondence to the ground truth caption. However, in various cases, it is quite possible that two captions which have different words or structure, might carry the same meaning and vice versa.…”

Section: Qualitative Analysismentioning

confidence: 99%

See 1 more Smart Citation

SubICap: Towards Subword-informed Image Captioning

Sharif¹,

Bennamoun²,

Liu³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Existing Image Captioning (IC) systems model words as atomic units in captions and are unable to exploit the structural information in the words. This makes representation of rare words very difficult and out-of-vocabulary words impossible. Moreover, to avoid computational complexity, existing IC models operate over a modest sized vocabulary of frequent words, such that the identity of rare words is lost. In this work we address this common limitation of IC systems in dealing with rare words in the corpora. We decompose words into smaller constituent units 'subwords' and represent captions as a sequence of subwords instead of words. This helps represent all words in the corpora using a significantly lower subword vocabulary, leading to better parameter learning. Using subword language modeling, our captioning system improves various metric scores, with a training vocabulary size approximately 90% less than the baseline and various state-of-the-art word-level models. Our quantitative and qualitative results and analysis signify the efficacy of our proposed approach.

show abstract

Section: Quantitative Analysismentioning

confidence: 68%

Section: Qualitative Analysismentioning

confidence: 99%

SubICap: Towards Subword-informed Image Captioning

Sharif¹,

Bennamoun²,

Liu³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A number of automatic evaluation metrics have been proposed for captioning, which can be categorized into supervised [15], [16], [17] and unsupervised [18], [19], [20] methods. The work presented here, falls in the latter category.…”

Section: • Automatic Evaluation Metricsmentioning

confidence: 99%

“…Moreover, comparing visual (source) and textual (target) information is not a straightforward task, and greatly adds to the complexity. -Supervised Metrics: Supervised metrics such as NNEval [15] and LCEval [16] combine existing metrics into a single unified measure, which has shown improvement in performance. However, the drawbacks of learned metrics are their high complexity and subjectivity to the training examples.…”

Section: Calculate Similaritymentioning

confidence: 99%

WEmbSim: A Simple yet Effective Metric for Image Captioning

Sharif¹,

White²,

Bennamoun³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

The area of automatic image caption evaluation is still undergoing intensive research to address the needs of generating captions which can meet adequacy and fluency requirements. Based on our past attempts at developing highly sophisticated learning-based metrics, we have discovered that a simple cosine similarity measure using the Mean of Word Embeddings (MOWE) of captions can actually achieve a surprisingly high performance on unsupervised caption evaluation. This inspires our proposed work on an effective metric WEmbSim, which beats complex measures such as SPICE, CIDEr and WMD at system-level correlation with human judgments. Moreover, it also achieves the best accuracy at matching human consensus scores for caption pairs, against commonly used unsupervised methods. Therefore, we believe that WEmbSim sets a new baseline for any complex metric to be justified.

show abstract

“…It learns the most predictive features (learned features) directly from data given a large dataset of labeled examples. In recent years, deep learning techniques have emerged as highly effective methods for prediction and decision-making in a multitude of disciplines including health (hearing aids), computer vision (e.g., object and face identification), [2], [3], [4], [5], natural language processing [6], [7], [8], gesture recognition [9], [10], [11], and robotics [12].…”

Section: Introductionmentioning

confidence: 99%

Deep Learning Models for Early Detection and Prediction of the spread of Novel Coronavirus (COVID-19)

Ayris¹,

Horbury²,

Williams³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

SARS-CoV2, which causes coronavirus disease (COVID-19) is continuing to spread globally and has become a pandemic. People have lost their lives due to the virus and the lack of counter measures in place. Given the increasing caseload and uncertainty of spread, there is an urgent need to develop machine learning techniques to predict the spread of COVID-19. Prediction of the spread can allow counter measures and actions to be implemented to mitigate the spread of COVID-19. In this paper, we propose a deep learning technique, called Deep Sequential Prediction Model (DSPM) and machine learning based Non-parametric Regression Model (NRM) to predict the spread of COVID-19. Our proposed models were trained and tested on publicly available novel coronavirus 2019 dataset. The proposed models were evaluated by using Mean Absolute Error and compared with baseline method. Our experimental results, both quantitative and qualitative, demonstrate the superior prediction performance of the proposed models.

show abstract

LCEval: Learned Composite Metric for Caption Evaluation

Cited by 10 publications

References 43 publications

SubICap: Towards Subword-informed Image Captioning

SubICap: Towards Subword-informed Image Captioning

WEmbSim: A Simple yet Effective Metric for Image Captioning

Deep Learning Models for Early Detection and Prediction of the spread of Novel Coronavirus (COVID-19)

Contact Info

Product

Resources

About