The SFU Opinion and Comments Corpus: A Corpus for the Analysis of Online News Comments

Abstract

We present the SFU Opinion and Comments Corpus (SOCC), a collection of opinion articles and the comments posted in response to the articles. The articles include all the opinion pieces published in the Canadian newspaper The Globe and Mail in the 5-year period between 2012 and 2016, a total of 10,339 articles and 663,173 comments. SOCC is part of a project that investigates the linguistic characteristics of online comments. The corpus can be used to study a host of pragmatic phenomena. Among other aspects, researchers can explore: the connections between articles and comments; the connections of comments to each other; the types of topics discussed in comments; the nice (constructive) or mean (toxic) ways in which commenters respond to each other; how language is used to convey very specific types of evaluation; and how negation affects the interpretation of evaluative meaning in discourse. Our current focus is the study of constructiveness and evaluation in the comments. To that end, we have annotated a subset of the large corpus (1043 comments) with four layers of annotations: constructiveness, toxicity, negation and Appraisal (Martin and White, The language of evaluation, Palgrave, New York, 2005). This paper details our corpus, the data collection process, the characteristics of the corpus and describes the annotations. While our focus is comments posted in response to opinion news articles, the phenomena in this corpus are likely to be present in many commenting platforms: other news comments, comments and replies in fora such as Reddit, feedback on blogs, or YouTube comments.

Studying Generalisability across Abusive Language Detection Datasets

“…(The differences are all statistically significant when the test set is Waseem and Hovy.) Similar trends were observed by Karan andŠnajder (2018) when employing the Kolhatkar et al (2018) and TRAC-1 (Kumar et al, 2018a) datasets, that have 62.7% and 56.6% positive samples, respectively, and exhibited better results in cross-dataset testing than datasets with lower positive sample ratios. 6 Synthesising Subtasks Using the Hierarchical Model…”

The interplay of complexity and subjectivity in opinionated discourse

“…Thus, a direct comparison between different types of opinionated discourse – the more formal opinion articles versus the comparatively more personal and informal reader comments – is facilitated. The data spans the time period from 2012 to 2016 and was collected from the Canadian online newspaper The Globe and Mail (Kolhatkar et al, 2020). SOCC has been specifically designed to analyse the characteristics of online comments preserving the comments’ thread-structure and providing extensive discourse annotation (e.g.…”

Cross-Domain Detection of Abusive Language Online

“…A bewildering plethora of different types of abusive language can be found online. Some of the types dealt with in related work include but are not limited to sexism, racism (Waseem and Hovy, 2016;Waseem, 2016), toxicity (Kolhatkar et al, 2018), hatefulness (Gao and Huang, 2017), aggression (Kumar et al, 2018), attack (Wulczyn et al, 2017), obscenity, threats, and insults. A typology of abusive language detection subtasks was recently proposed by Waseem et al (2017).…”