Inferring Values via Hybrid Intelligence

Liscio, Enrico; Lera-Leri, Roger; Bistaffa, Filippo; Dobbe, Roel I. J.; Jonker, Catholijn M.; Lopez-Sanchez, Maite; Rodriguez-Aguilar, Juan A.; Murukannaiah, Pradeep K.

doi:10.3233/faia230102

Cited by 1 publication

(1 citation statement)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Disagreement is a complex signal that includes subjective views, task understanding, and annotator behavior (Aroyo and Welty, 2015). Having access to non-aggregated annotations would, e.g., allow for further modeling of patterns (Davani et al, 2022) or the reasons (Liscio et al, 2023) underlying opinions. We investigate whether models trained on such annotations can identify disagreement.…”

Section: Modeling Diversity In Key Point Analysismentioning

confidence: 99%

HyEnA: A Hybrid Method for Extracting Arguments from Opinions

Meer

Liscio

Jonker

et al. 2022

HHAI2022: Augmenting Human Intellect

View full text Add to dashboard Cite

The key arguments underlying a large and noisy set of opinions help understand the opinions quickly and accurately. Fully automated methods can extract arguments but (1) require large labeled datasets and (2) work well for known viewpoints, but not for novel points of view. We propose HyEnA, a hybrid (human + AI) method for extracting arguments from opinionated texts, combining the speed of automated processing with the understanding and reasoning capabilities of humans. We evaluate HyEnA on three feedback corpora. We find that, on the one hand, HyEnA achieves higher coverage and precision than a state-of-the-art automated method, when compared on a common set of diverse opinions, justifying the need for human insight. On the other hand, HyEnA requires less human effort and does not compromise quality compared to (fully manual) expert analysis, demonstrating the benefit of combining human and machine intelligence.

show abstract

Section: Modeling Diversity In Key Point Analysismentioning

confidence: 99%