Critical reflections on three popular computational linguistic approaches to examine Twitter discourses

Abstract

Although computational linguistic methods—such as topic modelling, sentiment analysis and emotion detection—can provide social media researchers with insights into online public discourses, it is not inherent as to how these methods should be used, with a lack of transparent instructions on how to apply them in a critical way. There is a growing body of work focusing on the strengths and shortcomings of these methods. Through applying best practices for using these methods within the literature, we focus on setting expectations, presenting trajectories, examining with context and critically reflecting on the diachronic Twitter discourse of two case studies: the longitudinal discourse of the NHS Covid-19 digital contact-tracing app and the snapshot discourse of the Ofqual A Level grade calculation algorithm, both related to the UK. We identified difficulties in interpretation and potential application in all three of the approaches. Other shortcomings, such the detection of negation and sarcasm, were also found. We discuss the need for further transparency of these methods for diachronic social media researchers, including the potential for combining these approaches with qualitative ones—such as corpus linguistics and critical discourse analysis—in a more formal framework.

Special issue on analysis and mining of social media data

“…Four studies focused on tackling online harms of different kinds, with studies on abusive language detection ( Almerekhi, Kwak & Jansen, 2022 ; Ramponi et al., 2022 ), suicidal ideation detection ( Baghdadi et al., 2022 ) and misinformation detection ( Obeidat et al., 2022 ). Others studied NLP techniques for social media , focused on the analysis of Twitter discourse ( Heaton et al., 2023 ), language identification ( Hidayatullah et al., 2023 ) and named entity recognition ( Fudholi et al., 2023 ). There are also a number of analytical studies that investigate different societal issues, including gender equality through advertising data ( Al Tamime & Weber, 2022 ), analysis of gender-based violence across countries ( Rimjhim & Dandapat, 2022 ) as well as electoral data ( Yang, Hui & Menczer, 2022 ).…”

“… Heaton et al. (2023) made a critical reflection on how techniques for computational linguistics are being used to analyse Twitter discourse with different research objectives in mind.…”

“The algorithm will screw you”: Blame, social actors and the 2020 A Level results algorithm on Twitter

“…To demonstrate the need to combine the aforementioned analytical approaches, an overview of limitations affecting sentiment analysis, among other approaches, will follow. This is set up in the context of the previous Heaton et al study [ 5 ]. Additionally, an outline of CL and CDA—the chosen approaches—will be used to review existing contributions, which used similar methods to investigate Twitter discourses.…”

“…As previously mentioned, Heaton et al used sentiment analysis and other NLP-based computational linguistic tools to examine the views expressed on Twitter regarding the Ofqual algorithm, [ 5 ], critically evaluating sentiment analysis, topic modelling and emotion detection tools for textual analysis purposes.…”

“…As indicated previously, only one study has taken social media responses into account when considering the public reaction to the algorithm [ 5 ]. However, the NLP-based approach, as well as issues with accuracy and interpretation, meant that the contribution did not explore social actors and who was therefore blamed.…”