A Classification of Grammar-Infused Templates for Ontology and Model Verbalisation

Mahlaza, Zola; Keet, C. Maria

doi:10.1007/978-3-030-36599-8_6

Cited by 2 publications

(1 citation statement)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The ten best performing systems in the E2E challenge, based on the normalised average of automated metrics (BLEU, NIST, METEOR, ROUGE-L, and CIDEr), all use data-driven approaches. The best performing systems in the WebNLG challenge, based on automated metrics (BLUE, TER, and METEOR), are largely data-driven-the exception being the METEOR metric comparisons where a template system enhanced with some grammar (also called 'grammar-infused' [6]) outperforms all systems.…”

Section: Related Workmentioning

confidence: 99%

An evaluation of template and ML-based generation of user-readable text from a knowledge graph

Mahlaza,

Keet,

Dunn

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Typical user-friendly renderings of knowledge graphs are visualisations and natural language text. Within the latter HCI solution approach, data-driven natural language generation systems receive increased attention, but they are often outperformed by template-based systems due to suffering from errors such as content dropping, hallucination, or repetition. It is unknown which of those errors are associated significantly with low quality judgements by humans who the text is aimed for, which hampers addressing errors based on their impact on improving human evaluations. We assessed their possible association with an experiment availing of expert and crowdsourced evaluations of human authored text, template generated text, and sequence-to-sequence model generated text. The results showed that there was no significant association between human authored texts with errors and the low human judgements of naturalness and quality. There was also no significant association between machine learning generated texts with dropped or hallucinated slots and the low human judgements of naturalness and quality. Thus, both approaches appear to be viable options for designing a natural language interface for knowledge graphs

show abstract

Section: Related Workmentioning

confidence: 99%