Author’s Sentiment Prediction

Bastan, Mohaddeseh; Koupaee, Mahnaz; Son, Youngseo; Sicoli, Richard; Balasubramanian, Niranjan

doi:10.18653/v1/2020.coling-main.52

“…Bastan et al (2020) investigates the reverse direction, i.e., from paragraph-level predictions to document-level predictions.6 Similar trends hold if we adjust the estimates using TPR, TNR, FPR, and FNR. See the appendix.…”

mentioning

confidence: 82%

On Positivity Bias in Negative Reviews

Aithal

¹

,

Tan

²

2021

Preprint

2

1

0

View full text Add to dashboard Cite

Prior work has revealed that positive words occur more frequently than negative words in human expressions, which is typically attributed to positivity bias, a tendency for people to report positive views of reality. But what about the language used in negative reviews? Consistent with prior work, we show that English negative reviews tend to contain more positive words than negative words, using a variety of datasets. We reconcile this observation with previous findings on the pragmatics of negation, and show that negations are commonly associated with positive words in negative reviews. Furthermore, in negative reviews, the majority of sentences with positive words express negative opinions based on sentiment classifiers, indicating some form of negation.1 Boucher and Osgood (1969) used a morphological analysis to show negative affixes are more commonly applied to positive words than negative words (unhappy vs. non-violent).

show abstract

“…On grammaticality, there is some disagreement in judgment 0.7% of the time, while there is some disagreement in judgment 1% of the time for answer validity. We measured the inter-rater reliability of annotators' judgments using weighted Fleiss's Kappa (Marasini et al, 2016) and follow the weighting scheme used by Bastan et al (2020). This measure has a penalty for each dissimilar classification based on the distance between two classes.…”

Section: Validating Answersmentioning

confidence: 99%

“…The weights between different classes are shown in Table 6 where negative, slightly negative, neutral, slightly positive, and positive classes are shown with -2, -1, 0, 1, and 2. We follow the setup used in Bastan et al (2020) for a similar multi-class labeling task.…”

Section: C2 Inter-annotator Agreementmentioning

confidence: 99%

TellMeWhy: A Dataset for Answering Why-Questions in Narratives

Lal¹,

Chambers²,

Mooney³

et al. 2021

Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021

Self Cite

View full text Add to dashboard Cite

Answering questions about why characters perform certain actions is central to understanding and reasoning about narratives. Despite recent progress in QA, it is not clear if existing models have the ability to answer "why" questions that may require commonsense knowledge external to the input narrative. In this work, we introduce TellMeWhy, a new crowd-sourced dataset that consists of more than 30k questions and free-form answers concerning why characters in short narratives perform the actions described. For a third of this dataset, the answers are not present within the narrative. Given the limitations of automated evaluation for this task, we also present a systematized human evaluation interface for this dataset. Our evaluation of state-of-the-art models show that they are far below human performance on answering such questions. They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.

show abstract

“…The weights between different classes are shown in Table 6 where negative, slightly negative, neutral, slightly positive, and positive classes are shown with -2, -1, 0, 1, and 2. We follow the setup used in Bastan et al (2020)…”

Section: A Hyperparameter Sweepmentioning

confidence: 99%

TellMeWhy: A Dataset for Answering Why-Questions in Narratives

Lal,

Chambers,

Mooney

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

Answering questions about why characters perform certain actions is central to understanding and reasoning about narratives. Despite recent progress in QA, it is not clear if existing models have the ability to answer "why" questions that may require commonsense knowledge external to the input narrative. In this work, we introduce TellMeWhy, a new crowd-sourced dataset that consists of more than 30k questions and free-form answers concerning why characters in short narratives perform the actions described. For a third of this dataset, the answers are not present within the narrative. Given the limitations of automated evaluation for this task, we also present a systematized human evaluation interface for this dataset. Our evaluation of state-of-the-art models show that they are far below human performance on answering such questions. They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.

show abstract

Author’s Sentiment Prediction

Cited by 11 publications

References 31 publications

On Positivity Bias in Negative Reviews

On Positivity Bias in Negative Reviews

TellMeWhy: A Dataset for Answering Why-Questions in Narratives

TellMeWhy: A Dataset for Answering Why-Questions in Narratives

Contact Info

Product

Resources

About