The utility of topic modelling for discourse studies: A critical evaluation

Abstract

This article explores and critically evaluates the potential contribution to discourse studies of topic modelling, a group of machine learning methods which have been used with the aim of automatically discovering thematic information in large collections of texts. We critically evaluate the utility of the thematic grouping of texts into 'topics' emerging from a large collection of online patient comments about the National Health Service (NHS) in England. We take two approaches to this, one inspired by methods adopted in existing topic modelling research and the other using more established methods of discourse analysis. In the study, we compare the insights produced by each approach and consider the extent to which the automatically generated topics might be of use to discourse analysts attempting to organise and study sizeable datasets. We found that the topic modelling approach was able to group texts into 'topics' that were truly thematically coherent with a mixed degree of success, while the more traditional approach to discourse analysis consistently provided a more nuanced perspective on the data which was ultimately closer to the 'reality' of the texts it contains. This study thus highlights issues concerning the use of topic modelling and offers recommendations and caveats to researchers employing such approaches to studying discourse in the future.

‘So is your mom as cute as you?’: Examining patterns of language use in online sexual grooming of children

“…Our own view is that a groomer's degree of influence online over his/her target cannot be simply measured through static linguistic features, such as particular pronoun usage. Rather, digital influence is a more complex and dynamic process (Lorenzo-Dus and Di Cristofaro, 2016) -one that, like topic/theme identification in large corpora, operates at a broader contextual level (see Van Dijk, 1980;and Brookes and McEnery (2018) for a critical evaluation of topic modelling approaches).…”

“…Doing so on the basis of automated (software-enabled), decontextualized, single word analysis -as enabled by the LIWC software -disregards the proven link between the textual context in which words are used and their meaning. We share Brookes and McEnery's (2018) view, in relation to the use of topic modelling approaches, that quantitative, largely decontextualized, single word based methodologies 'should not replace close, qualitative analysis of the texts in the data ' (2018, p. 17). Instead, an integration of automated and manual methods should be adopted, to be guided by specific research questions.…”

Notícias relacionadas a pessoas com deficiência: uma análise do conteúdo gerado pelos usuários em postagens de mídias sociais

“…Foram adotadas diferentes composições para testes nos tópicos mais utilizados nas postagens. O objetivo destas composiçõesé rotular, comparar e selecionar a melhor configuração, dada que a escolha desses parâmetrosé feita arbitrariamente [Brookes and McEnery 2019]. As principais modificações deram-se sobre a quantidade de tópicos (T) e de palavras (P).…”

Uma abordagem baseada em letras para a descoberta de conhecimento da música brasileira: o sertanejo como um estudo de caso

“…Ao alternar entre esses esquemas notou-se uma grande dispersão, como a ocorrência repetitiva de tópicos ou palavras; ou então as composições eram insuficientes para a rotulação pelos anotadores. Dado que a escolha desses esquemasé arbitrária [Brookes andMcEnery 2019, Rodrigues et al 2019], procurou-se a composição que melhor permitiria a facilidade de anotação dos tópicos. Nesse estudo a análise da modelagem de tópicos consiste em uma análise manual das palavras mais representativas de cada tópico.…”