Argumentation models and their use in corpus annotation: Practice, prospects, and challenges

Abstract

The study of argumentation is transversal to several research domains, from philosophy to linguistics, from the law to computer science and artificial intelligence. In discourse analysis, several distinct models have been proposed to harness argumentation, each with a different focus or aim. To analyze the use of argumentation in natural language, several corpora annotation efforts have been carried out, with a more or less explicit grounding on one of such theoretical argumentation models. In fact, given the recent growing interest in argument mining applications, argument-annotated corpora are crucial to train machine learning models in a supervised way. However, the proliferation of such corpora has led to a wide disparity in the granularity of the argument annotations employed. In this paper, we review the most relevant theoretical argumentation models, after which we survey argument annotation projects closely following those theoretical models. We also highlight the main simplifications that are often introduced in practice. Furthermore, we glimpse other annotation efforts that are not so theoretically grounded but instead follow a shallower approach. It turns out that most argument annotation projects make their own assumptions and simplifications, both in terms of the textual genre they focus on and in terms of adapting the adopted theoretical argumentation model for their own agenda. Issues of compatibility among argument-annotated corpora are discussed by looking at the problem from a syntactical, semantic, and practical perspective. Finally, we discuss current and prospective applications of models that take advantage of argument-annotated corpora.

Automatic Analysis of Substantiation in Scientific Peer Reviews

“…Different from Toulmin's model, it proposes to analyze arguments as product rather than as process. As a result, it is more applicable to real-life arguments and commonly exploited in computational linguistics settings (Lopes Cardoso et al, 2023).…”

“…More results from this annotation round can be found in Appendix B. We believe that the unsatisfactory IAA is due to the high number of annotators which naturally amplifies the bias (Lopes Cardoso et al, 2023). Moreover, the annotators only went through a 2 hour training session, thus not qualifying as experts for this task.…”

Brakujące elementy argumentów w debacie konkursowej

“…Przedstawione rekonstrukcje były bowiem nie tylko próbą zinterpretowania tego, co dzieje się w debatach konkursowych i jaka jest rola przesłanek entymematycznych, lecz także przymiarką do opracowania schematu anotacji dla całego korpusu tekstów. Jak pokazują najnowsze opracowania (Harrell 2022;Lopes i in. 2023), mimo istnienia różnorodnych sposobów wizualizacji debaty i modeli argumentacji nie widać dominującego standardu, który pozwalałby kompleksowo ujmować zjawiska ze skali mikro i makro, powierzchniowe (tekstowe) i głębokie (logiczne).…”