Aspect-Controlled Neural Argument Generation

Schiller, Benjamin; Daxenberger, Johannes; Gurevych, Iryna

doi:10.18653/v1/2021.naacl-main.34

Cited by 40 publications

(41 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Leveraging external knowledge, though a promising feature for guiding finetuning, may benefit from better encoding strategies compared to the conventional method of using control codes in text. However, given that the identified knowledge is extractive and that we encoded multiple aspects and targets per example in contrast to related controlled text generation approaches Schiller et al, 2020;Gretz et al, 2020;Cachola et al, 2020), further investigations with importance sampling of argumentative knowledge are advised. Ideally, such sampling would be tailored to a specific domain or target audience.…”

Section: Discussionmentioning

confidence: 99%

“…Gretz et al (2020) proposed a pipeline based on GPT-2 (Radford et al, 2019) for generating coherent claims for a given debate topic. A more controlled approach for argument generation was developed by Schiller et al (2020), which performs argument generation with fine-grained control of topic, aspect (core reasoning), and stance. Conclusion generation can be viewed as supplementing argument generation.…”

Section: Related Workmentioning

confidence: 99%

“…Aspects: Text spans that contribute to the core reasoning of an argument are called its aspects (Schiller et al, 2020). Aspects can be viewed as subtopics related to the main topic of an argumentative text, encoding a stance.…”

Section: Encoded Representationmentioning

confidence: 99%

See 2 more Smart Citations

Generating Informative Conclusions for Argumentative Texts

Syed,

Al-Khatib,

Alshomary

et al. 2021

Preprint

View full text Add to dashboard Cite

The purpose of an argumentative text is to support a certain conclusion. Yet, they are often omitted, expecting readers to infer them rather. While appropriate when reading an individual text, this rhetorical device limits accessibility when browsing many texts (e.g., on a search engine or on social media). In these scenarios, an explicit conclusion makes for a good candidate summary of an argumentative text. This is especially true if the conclusion is informative, emphasizing specific concepts from the text. With this paper we introduce the task of generating informative conclusions: First, Webis-ConcluGen-21 is compiled, a large-scale corpus of 136,996 samples of argumentative texts and their conclusions. Second, two paradigms for conclusion generation are investigated; one extractive, the other abstractive in nature. The latter exploits argumentative knowledge that augment the data via control codes and finetuning the BART model on several subsets of the corpus. Third, insights are provided into the suitability of our corpus for the task, the differences between the two generation paradigms, the trade-off between informativeness and conciseness, and the impact of encoding argumentative knowledge. The corpus, code, and the trained models are publicly available. 1

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Generating Informative Conclusions for Argumentative Texts

Syed,

Al-Khatib,

Alshomary

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

“…However, these control codes, derived from the structure that naturally co-occurs with raw texts, cover constrained controllable attributes. Following works like Arg-CTRL [41] and Tailor [42] extended the control codes of CTRL either by crowdsourcing aspect annotations or deriving from the PropBank formalism to control the semantic-specific generation. Another line of work [43,44] aims to generate texts of desired attributes through relatively small 'pluggable' attribute models focusing on light-weight controllable fine-tuning on pre-trained models.…”

Section: Controllable Text Generationmentioning

confidence: 99%

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Wang¹,

Sun²,

Xiang³

et al. 2021

Preprint

View full text Add to dashboard Cite

Pre-trained language models have achieved state-of-the-art results in various Natural Language Processing (NLP) tasks. has shown that scaling up pre-trained language models can further exploit their enormous potential. A unified framework named ERNIE 3.0 [2] was recently proposed for pre-training large-scale knowledge enhanced models and trained a model with 10 billion parameters. ERNIE 3.0 outperformed the state-of-the-art models on various NLP tasks. In order to explore the performance of scaling up ERNIE 3.0, we train a hundred-billion-parameter model called ERNIE 3.0 Titan with up to 260 billion parameters on the PaddlePaddle [3] platform. Furthermore, we design a self-supervised adversarial loss and a controllable language modeling loss to make ERNIE 3.0 Titan generate credible and controllable texts. To reduce the computation overhead and carbon emission, we propose an online distillation framework for ERNIE 3.0 Titan, where the teacher model will teach students and train itself simultaneously. ERNIE 3.0 Titan is the largest Chinese dense pre-trained model so far. Empirical results show that the ERNIE 3.0 Titan outperforms the state-of-the-art models on 68 NLP datasets.

show abstract

“…Extracting key points is conceptually similar to identifying aspects (Bar-Haim et al, 2020a), which inspired our clustering approach that selects representative sentences from multiple aspect clusters as the final key points. We employ the tagger of Schiller et al (2021) to extract the arguments' aspects (on average, 2.1 aspects per argument). To tackle the lack of diversity, we follow Heinisch and Cimiano (2021) final set of key points summarizing the arguments for a given topic and stance maximizes the coverage of the set of arguments' aspects.…”

Section: Aspect Clusteringmentioning

confidence: 99%

Key Point Analysis via Contrastive Learning and Extractive Argument Summarization

Alshomary¹,

Gurcke²,

Syed³

et al. 2021

Preprint

View full text Add to dashboard Cite

Key point analysis is the task of extracting a set of concise and high-level statements from a given collection of arguments, representing the gist of these arguments. This paper presents our proposed approach to the Key Point Analysis shared task, collocated with the 8th Workshop on Argument Mining. The approach integrates two complementary components. One component employs contrastive learning via a siamese neural network for matching arguments to key points; the other is a graph-based extractive summarization model for generating key points. In both automatic and manual evaluation, our approach was ranked best among all submissions to the shared task.

show abstract

Aspect-Controlled Neural Argument Generation

Cited by 40 publications

References 40 publications

Generating Informative Conclusions for Argumentative Texts

Generating Informative Conclusions for Argumentative Texts

ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation

Key Point Analysis via Contrastive Learning and Extractive Argument Summarization

Contact Info

Product

Resources

About