Opportunities and Challenges for AI-Assisted Qualitative Data Analysis: An Example from Collaborative Problem-Solving Discourse Data

Siiman, Leo; Rannastu-Avalos, Meeli; Pöysä-Tarhonen, Johanna; Häkkinen, Païvi; Pedaste, Margus

doi:10.1007/978-3-031-40113-8_9

Cited by 11 publications

(1 citation statement)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As tested by researchers, Savelka et al (2023) claimed that GPT-4 can analyze textual data in the context of a task involving the interpretation of legal concepts at a level comparable to that of law student annotators who have received adequate training. Another study reported that GPT-4 can assist in qualitative data analysis, and careful prompt structuring can align AI responses closely with human interpretations, particularly in deductive analysis (Siiman, 2023). The previous findings suggested that there is a need to explore more about GPT-4's ability to analyze different types of textual data.…”

Section: Literature Reviewmentioning

confidence: 99%

A Comparison of Rhetorical Move Analysis by GPT-4 and Humans in Abstracts of Scopus-Indexed Tourism Research Articles

Geng,

Nimehchisalem,

Zargar

et al. 2024

ILR

View full text Add to dashboard Cite

AI advancements have made ChatGPT a remarkable and versatile tool in education and linguistics, showcasing its potential to mimic human conversation and comprehend language. Scholars are intrigued by ChatGPT’s text data handling, yet its application in rhetorical move analysis remains largely unexplored. Therefore, the objective of this study is to investigate the ability of GPT-4 in the identification of rhetorical moves employed in the abstracts of tourism research articles indexed in Scopus. The essentiality of moves was also reported. Additionally, this research seeks to compare the accuracy of GPT-4’s analysis with that of humans. Adopting Hyland’s (2000) five-move model, the results indicated that GPT-4 analyzes moves more quickly but less accurately than human experts, and the four principal types of errors committed by GPT-4 include redundancy/over-count, unmatched categorization, incorrect sequence, and vague identification. The findings also revealed that Move 2 (Purpose) and Move 4 (Findings) are obligatory with a 100% essentiality rate through both GPT-4 and human analysis. Differences arise in certain steps of Move 1 (Introduction), Move 3 (Methods), and Move 5 (Conclusion), where GPT-4 often sees higher essentiality rates. This study shed light on the testament to AI’s current capabilities in move analysis in academic discourse.

show abstract

Section: Literature Reviewmentioning

confidence: 99%