Providing coaching to participants in inquiry‐based online discussions contributes to developing cognitive presence (CP) and higher‐order thinking. However, a primary issue limiting quality and timely coaching is instructors' lack of tools to efficiently identify CP phases in massive discussion transcripts and effectively assess learners' cognitive development. This study examined a computational approach integrating text mining and co‐occurrence analysis for assessing CP and cognitive development in online discussions based on the community of inquiry (CoI) framework. First, text classifiers trained on different language models were evaluated for identifying and coding the CP phases. Second, epistemic network analysis (ENA) was employed to model cognitive patterns reflected by co‐occurrences between the coding elements. Results indicated that text classifiers trained on the state‐of‐the‐art language model Bidirectional Encoder Representations from Transformers (BERT) can address the efficiency issue in coding CP phases in discussion transcripts and obtain substantial agreements (Cohen's k = 0.76) with humans, which outperformed other baseline classifiers. Furthermore, compared to traditional quantitative content analysis, ENA can effectively model the temporal characteristics of online discourse and detect fine‐grained cognitive patterns. Overall, the findings suggest a feasible path for applying learning analytics to tracking learning progression and informing theory‐based assessments.
What is already known about this topic
Cognitive presence is an important construct describing the progression of thinking in online inquiry‐based discussions.
Most studies used self‐report instruments or quantitative content analysis to measure and assess cognitive presence.
More efficient and effective approaches were needed by instructors to support assessment of cognitive development and determine coaching strategies.
What this paper adds
An integrated computational approach for the developmental and formative assessment of cognitive presence was proposed and evaluated.
A BERT‐based text classification model could efficiently code massive transcripts and achieve substantial agreements with human coders.
Epistemic network analysis effectively revealed the process of cognitive development and identified representative discussion patterns and behaviours.
Implications for practice and/or policy
The proposed approach can considerably reduce the pressure on instructors, enabling them to focus on quality coaching and feedback.
Compared to frequencies of individual codes, the connective features between codes carry more insights for assessing cognitive patterns.
Learners in a discussion group play different roles and produce diverse paths of cognitive development.
Despite the continuous emphasis on emotion in multimedia learning, it was still unclear how pedagogical agent emotional cues might affect learning. In the present study, a between-subjects experiment was performed to examine the effects of a pedagogical agent with dual-channel emotional cues on learners' emotions, cognitive load, and knowledge transfer performance. Participants from a central Chinese university (age mean = 21.26, N = 66) were randomly divided into three groups. These groups received instructions from an affective pedagogical agent, a neutral pedagogical agent, or a neutral voice narration without pedagogical agent embodiment. Results showed that learners assigned the affective pedagogical agent reported a significantly higher emotional level than learners assigned the neutral pedagogical agent. Learners’ perceived task difficulty was not significantly different among groups while instructional efficiency was significantly higher for learners with the affective pedagogical agent. Moreover, learners assigned to the affective pedagogical agent performed significantly better on the knowledge transfer test than those assigned the neutral pedagogical agent or the neutral voice.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.