A Simple Recipe towards Reducing Hallucination in Neural Surface Realisation

Abstract

Recent neural language generation systems often hallucinate contents (i.e., producing irrelevant or contradicted facts), especially when trained on loosely corresponding pairs of the input structure and text. To mitigate this issue, we propose to integrate a language understanding module for data refinement with selftraining iterations to effectively induce strong equivalence between the input data and the paired text. Experiments on the E2E challenge dataset show that our proposed framework can reduce more than 50% relative unaligned noise from the original data-text pairs. A vanilla sequence-to-sequence neural NLG model trained on the refined data has improved on content correctness compared with the current state-of-the-art ensemble generator. * Contribution during internship at Microsoft.

Semantic Noise Matters for Neural Natural Language Generation

“…Contemporaneous with our work is the effort of Nie et al (2019), who focus on automatic data cleaning using a NLU iteratively bootstrapped from the noisy data. Their analysis similarly finds that omissions are more common than hallucinations.…”

Controlled Hallucinations: Learning to Generate Faithfully from Noisy Data

“…The former path is risky as it easily results in ungrammatical targets. The latter approach of enforcing a stronger alignment between inputs and outputs has been tried previously but it assumes a moderate amount of noise in the data (Nie et al, 2019;Dušek et al, 2019). Alternatively, one can leave the data as is and try to put more pressure on the decoder to pay attention to the input at every generation step (Tian et al, 2019).…”

A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction

“…It is worth noting that SR can be regarded as a variant of self-training due to its structural similarity, except that it takes the target sentences rather than the source sentences as input to the model. The algorithm itself is the key difference from existing methods based on selftraining (Wang, 2019;Nie et al, 2019;Xie et al, 2020).…”