Experimental Assessment of Aggregation Principles in Argumentation-Enabled Collective Intelligence

Abstract

On the Web, there is always a need to aggregate opinions from the crowd (as in posts, social networks, forums, etc.). Di erent mechanisms have been implemented to capture these opinions such as Like in Facebook, Favorite in Twi er, thumbs-up/down, agging, and so on. However, in more contested domains (e.g. Wikipedia, political discussion, and climate change discussion) these mechanisms are not su cient since they only deal with each issue independently without considering the relationships between di erent claims. We can view a set of con icting arguments as a graph in which the nodes represent arguments and the arcs between these nodes represent the defeat relation. A group of people can then collectively evaluate such graphs. To do this, the group must use a rule to aggregate their individual opinions about the entire argument graph. Here, we present the rst experimental evaluation of di erent principles commonly employed by aggregation rules presented in the literature. We use randomized controlled experiments to investigate which principles people consider be er at aggregating opinions under di erent conditions. Our analysis reveals a number of factors, not captured by traditional formal models, that play an important role in determining the e cacy of aggregation. ese results help bring formal models of argumentation closer to real-world application. CCS Concepts: •Computing methodologies →Nonmonotonic, default reasoning and belief revision; Multi-agent systems; •Applied computing →Law, social and behavioral science;

A model to support collective reasoning: Formalization, analysis and computational assessment

“…We observe that although the direct opinion about τ is rather negative (v 2 (τ ) = −0.5), the valuations for its descendants are diverse: while the valuation for s 1 is also rather negative (v 2 (s 1 ) = −1), and hence in line with τ , the valuations on the other descendants are rather positive (v 2 (s 2 ) = 1 and v 2 (s 3 ) = 0.5), and hence not in line with τ . Thus, at first sight 6 it would seem that the overall estimated opinion is not in line with the direct opinion.…”

“…This can be achieved by: (1) transforming the graph associated to the DRF into an acyclic B-hypergraph 10 , denoted by H( S, R, T ) (line 4); and (2) then performing the topological sorting over the B-hypergraph (line 5). Starting from the sentences without descendants, the algorithm computes aggregated valuations until reaching the statements in T (lines [5][6][7][8][9][10][11][12][13][14][15][16][17][18]. The calculation of the topological sorting for B-hypergraphs has been studied in [44] In real debates the number of opinions is usually significantly higher than the number of relationships, hence, the complexity of the calculation of aggregated valuations is given by O(|S| × |P |).…”

Collective argumentation: A survey of aggregation issues around argumentation frameworks

“…While such analysis can be done in terms of general foundational principles, we believe that such debate can be carried out in more pragmatic terms, so the reasonableness of one approach or the other will depend on the specific collective argumentation context under consideration (e.g. the framework-wise approach could be the Other miscellaneous [8,28,39,59,60,64] most reasonable one in the context of deliberative democracy, while the argument-wise approach could be the most reasonable one in the context of a debate among experts). The interaction of collective argumentation with SCT, JAT and GT stands out, in our opinion, as the most promising approach in the field.…”

“…In this regard, we can say that collective argumentation is concerned with the explicit or implicit aggregation of individual preferences among arguments in order to find collective opinions based on collectively supported reasons. Deliberative democracy is clearly reflected in this characterization [62], but models of argument aggregation can potentially be used for a wider range of applications covering, for instance, collective intelligence [8] and prediction markets [60].…”

“…Awad, Bonnefon, Caminada, Malone, and Rahwan [8] show that formal models do not capture a number of factors that people consider important in real-life applications. In their work, the authors contrast two aggregation mechanisms, argument-wise plurality rule (AWPR) and skeptical and super credulous operators (SSCOs), the first one based on the idea that an assessment of an argument (in/out/undecided) is chosen if it is submitted by the majority regardless what the minority think, and the later one supporting the idea that minority's opinion should not be completely ignored.…”