Towards automated content analysis of discussion transcripts

Structure Matters: Adoption of Structured Classification Approach in the Context of Cognitive Presence Classification

“…Where humans construct meaning from text using various inferences and abstractions that manifest as complex higher-order cognitive processes, machine learning approaches require meticulously constructed feature spaces, which are representative of the problem task. Kovanović et al [17] presented an approach to classifying cognitive presence from online discussions, using a Support Vector Machine (SVM) classification model, which achieved classification accuracy of 58.84%. While the results of this work are promising, the overall performance of this approach is substantially less accurate than what can be achieved by human coders, which provides further evidence of the overall complexity of this task.…”

“…While the results of this work are promising, the overall performance of this approach is substantially less accurate than what can be achieved by human coders, which provides further evidence of the overall complexity of this task. In this approach, Kovanović et al [17] made use of lexical features derived from the content of each individual discussion message that are prominent within the literature. These features consisted of various N-grams, POS tags, name entity counts and dependency tuples, as well as intuitive features such as whether a post or reply is the first in a discussion thread.…”

“…One of the biggest practical challenges in adoption of the CoI framework -and other transcript analysis methods -is that it requires experienced coders and substantial labor-intensive work to code (i.e. categorise) discussion messages for the levels of three presences [17,4]. As such, it is argued that this and similar approaches have had very little practical impact upon current educational practices [4].…”

“…This approach i) does not enable for a real-time feedback on the quality of learning interactions, and ii) limits the wider adoption of the CoI framework by educational practitioners. This problem makes the task an ideal candidate for automation, and a number of approaches aimed at automating the process of coding transcripts using machine learning techniques are in development [22,2,17]. While these approaches have produced promising results, their text classification models currently make class predictions on a per-message basis, using only features derived from a single post, without consideration of the context of a post or of the preceding classification sequence.…”

Classifying Learners’ Cognitive Engagement From Online Discussion Using Text Mining

“…On the other side Kovanovic & all uses LIWC and Coh-Matrix to automate the coding of messages in order to deduce the cognitive presence [12], this method shows its efficiency in classification but only for a small number of thread-based context Features which can cause generalization issues.…”