Exploring the potential of natural language processing to support microgenetic analysis of collaborative learning discussions

Abstract

In this study, we explore the potential of a natural language processing (NLP) approach to support discourse analysis of in‐situ, small group learning conversations. The theoretical basis of this work derives from Bakhtin’s notion of speech genres as bounded by educational robotics activity. Our goal is to leverage computational linguistics methods to advance and improve educational research methods. We used a parts‐of‐speech (POS) tagging program to automatically parse a transcript of spoken dialogue collected from a small group of middle school students involved in solving a robotics challenge. We grammatically parsed the dialogue at the level of the trigram. Then, through a deliberative process, we mapped the POS trigrams to our theoretically derived problem solving in computational environments coding system. Next, we developed a stacked histogram visualization to identify rich interactional segments in the data. Seven segments of the transcript were thus identified for closer analysis. Our NLP‐based approach partially replicated prior findings. Here, we present the theoretical basis for the work, our analytical approach in exploring this NLP‐based method, and our research findings. What is already known about this topic Over the last 10 years, several educational research papers indicate that natural language processing (NLP) techniques can be used to help interpret well‐structured, written dialogue, eg, conversations in online class discussions. Two recent papers indicate that NLP techniques can also be used to help interpret well‐structured, spoken dialogue, eg, replies to interview questions and/or comments made during think aloud protocols. Multimodal learning analytic techniques are being used to investigate collaborative learning. These studies use non‐verbal features of data (gaze, gesture, physical actions), prosodic features of verbal data (pitch and tone) and/or turn‐taking and duration of talk per speaker data, as means of predicting group success. None of the MMLA studies attempt semantic analysis of student talk in collaborative settings. What this paper adds A theoretical framework for why and how an automated NLP approach can support discourse analysis research on co‐located, computer‐based, collaborative problem solving interactions. This framework, entitled the Problem Solving in Computational Environment Speech Genre, links children’s physical interactions with computational devices to their verbal exchanges and presents a theoretical rationale for the use of NLP methods in educational research. Description of an interdisciplinary method that combines NLP techniques with qualitative coding approaches to support analysis of student collaborative learning with educational robotics. Identification of student learning outcomes derived from the semantic, PSCE Speech Genre and NLP approach. Implications for practice and/or policy Educational researchers will be able to expand upon our findings towards the goal of using computation and automation to support microgenetic analysis of la...

Understanding the focal points and sentiment of learners in MOOC reviews: A machine learning and SC‐LIWC‐based approach

“…The authors further combined the measures derived from the discourse analysis and social ties to predict the student learning outcomes. Sullivan and Keith (2019) combined a NLP method with qualitative coding to support the discourse analysis in a collaborative learning setting and identify the student learning outcomes. Tucker, Pursel and Divinsky (2014) investigated the relationship between the student sentiment (expressed textually in MOOCs and quantified using advanced data mining techniques) and student performance in the course.…”

Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments

“…For example, Dowell et al (2020) applied a computational linguistic framework to analyze the sequential interactions of online team communication and to detect roles in regulation, social coordination, and meaning-making in discussion. In addition, Sullivan and Keith (2019) used a parts-ofspeech (POS) tagging program to automatically parse a transcript of spoken dialogue collected from a small group of middle school students involved in solving a robotics challenge. They grammatically analyzed the dialogue at the level of the tri-gram.…”

“…Identifying key phrases as relevant for types of regulation of CPS may help us in developing feedback mechanisms based on the recognition of applied phrases. In this study, we seek to extend our prior human-coding approaches to discourse analysis to better understand how finer-grained aspects of the group regulation in CPS influences computational modelling by utilizing natural language processing (NLP; Rosé et al, 2008), and specifically the text mining approach, Part-of-speech (POS) n-grams (Sullivan & Keith, 2019). In our analysis, we focused on the following features (building on Rosé et al, 2008;Sullivan & Keith, 2019), available in the publicly downloadable version of Rapidminer, to exploit context for developing a more complete understanding of the student discourse.…”

Understanding user trust in artificial intelligence‐based educational systems: Evidence from China

“…AI-based systems are characterized by their autonomy, adaptability and interactivity (Richards & Dignum, 2019;Xu, Wijekumar, Ramirez, Hu, & Irey, 2019). AI techniques can capture and analyze learners' behavioral and psychological data, connect to knowledge networks (Sullivan & Keith, 2019), change customized learning plans based on learners' responses, rather than being a set procedure designed by some experts.…”