Comparing text mining and manual coding methods: Analysing interview data on quality of care in long-term care for older adults

Abstract

Objectives In long-term care for older adults, large amounts of text are collected relating to the quality of care, such as transcribed interviews. Researchers currently analyze textual data manually to gain insights, which is a time-consuming process. Text mining could provide a solution, as this methodology can be used to analyze large amounts of text automatically. This study aims to compare text mining to manual coding with regard to sentiment analysis and thematic content analysis. Methods Data were collected from interviews with residents (n = 21), family members (n = 20), and care professionals (n = 20). Text mining models were developed and compared to the manual approach. The results of the manual and text mining approaches were evaluated based on three criteria: accuracy, consistency, and expert feedback. Accuracy assessed the similarity between the two approaches, while consistency determined whether each individual approach found the same themes in similar text segments. Expert feedback served as a representation of the perceived correctness of the text mining approach. Results An accuracy analysis revealed that more than 80% of the text segments were assigned the same themes and sentiment using both text mining and manual approaches. Interviews coded with text mining demonstrated higher consistency compared to those coded manually. Expert feedback identified certain limitations in both the text mining and manual approaches. Conclusions and implications While these analyses highlighted the current limitations of text mining, they also exposed certain inconsistencies in manual analysis. This information suggests that text mining has the potential to be an effective and efficient tool for analysing large volumes of textual data in the context of long-term care for older adults.

A Comprehensive Approach to Capturing the Impact and Identifying Countermeasures of the COVID-19 Pandemic at Construction Sites in the Republic of Korea

“…Text mining reduces bias caused by researcher's subjectivity compared to manual content analysis or thematic analysis. Manual analysis is susceptible to the personal preferences, interpretations, and inferences of the researcher, whereas text mining results are based on algorithms, thus overcoming the reliability weaknesses of content analysis [54][55][56]. Using the word frequency of interview content, we assigned additional weights to analyze the responses of construction site managers during COVID-19 in the ROK [57].…”

“…Verbs such as "be", "have", "do", "take", and "happen" were excluded due to repetition in other types of interviews. Synonyms and related words were also organized as much as possible [55,56]. Subsequently, word frequency was determined and counted numerically to identify the words and factors valued by the respondents.…”

The effect of construction intensity on landscape preferences in Cuban tourist resorts

“…To evaluate the coding obtained with AI tools compared to manual coding, several criteria were defined based on a review of related literature (Hacking et al, 2023;Moreiro, 2002), as well as others of own authorship applicable to this particular case:…”

“…In contrast to other studies where criteria weighting was automated through computer programs (Hacking et al, 2023), with the expert criterion being an additional aspect, in this case the expert criterion served as a general weighting guideline for the rest of the criteria. This decision was made on the premise that a weighting approach based on in-depth knowledge of the subject is preferable to a mathematical weighting through specific software.…”

“…The former argue that automatic coding is more reliable, objective and efficient (in terms of time) than manual coding, while the latter argue that this technology is still underdeveloped (Graaf and Vossen, 2013). Nevertheless, recent studies have found a high level of accuracy and consistency in AI coding compared to manual coding (Hacking et al, 2023).…”