A Systematic Comparison of Methods for Finding Good Premises for Claims

Dumani, Lorik; Schenkel, Ralf

doi:10.1145/3331184.3331282

Cited by 11 publications

(12 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They apply a standard BM25F ranking model implemented on top of Lucene. In our prior work [12], we build on the work of Wachsmuth et al and systematically compared 196 methods for identification of similar claims by textual similarity, using a comparable large corpus of (claim, premise) pairs crawled from several debate portals. The results imply that matching similar claims to a query claim with Divergence from Randomness (DFR) [2] yields slightly better results than BM25 [24].…”

Section: Related Workmentioning

confidence: 99%

“…We used the dataset of our prior work [12] which consists of 63,250 claims and about 695,000 premises extracted from four debate portals. After clustering, the 63,250 claims were distributed over a total of 10,611 clusters.…”

Section: Dataset and Baselinementioning

confidence: 99%

“…In Sect. 6 we evaluate our approach with a large corpus [12] consisting of 63,250 claims and about 695,000 premises and compare it to a baseline system that uses BM25F. Section 7 concludes the paper and discusses ideas for future work.…”

Section: Introductionmentioning

confidence: 99%

“…We perform the clustering with the scripting language R and the packages stats and fastCluster 3. Note that we use DFR to find similar claims, but not to find similar premises, because we only have a study supporting the former[12]. Also, claims and premises differ in length as well as details and information[13].…”

mentioning

confidence: 99%

See 3 more Smart Citations

A Framework for Argument Retrieval

Dumani

Neumann

Schenkel

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Computational argumentation has recently become a fast growing field of research. An argument consists of a claim, such as "We should abandon fossil fuels", which is supported or attacked by at least one premise, for example "Burning fossil fuels is one cause for global warming". From an information retrieval perspective, an interesting task within this setting is finding the best supporting and attacking premises for a given query claim from a large corpus of arguments. Since the same logical premise can be formulated differently, the system needs to avoid retrieving duplicate results and thus needs to use some form of clustering. In this paper we propose a principled probabilistic ranking framework for premises based on the idea of tf-idf that, given a query claim, first identifies highly similar claims in the corpus, and then clusters and ranks their premises, taking clusters of claims as well as the stances of query and premises into account. We compare our approach to a baseline system that uses BM25F which we outperform even with a primitive implementation of our framework utilising BERT.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Dataset and Baselinementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

A Framework for Argument Retrieval

Dumani

Neumann

Schenkel

2020

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, ranking premises is a particularly challenging task. It is not su cient to consider only the text-based similarity of the premise to the query claim, as we have shown in our prior work [11]; instead, the similarity of the premise's original claim to the query claim plays an important role. In this paper, we now argue that the quality of the premise should also be taken into account.…”

mentioning

confidence: 99%

Quality-Aware Ranking of Arguments

Dumani

Schenkel

2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

Self Cite

View full text Add to dashboard Cite

Argument search engines identify, extract, and rank the most important arguments for and against a given controversial topic. A number of such systems have recently been developed, usually focusing on classic information retrieval ranking methods that are based on frequency information. An important aspect that has been ignored so far by search engines is the quality of arguments. We present a quality-aware ranking framework for arguments already extracted from texts and represented as argument graphs, considering multiple established quality measures. An extensive evaluation with a standard benchmark collection demonstrates that taking quality into account signi cantly helps to improve retrieval quality for argument search. We also publish a dataset in which arguments with respect to topics were tediously annotated by humans with three widely accepted argument quality dimensions.

show abstract