Dialogue Act Classification In Human-to-Human Tutorial Dialogues

CPS‐Rater: Automated Sequential Annotation for Conversations in Collaborative Problem‐Solving Activities

“…Despite that CRF has a broad spectrum of applications in NLP, it has not been widely used to classify collaboration skills from conversations in collaborative activities. The closest applications of this kind are the classification of the dialog acts from live chats and tutorial dialogues (S. N. Kim, Cavedon, & Baldwin, ; Rus, Niraula, Maharjan, & Banjade, ) and the identification of the affects from human–human interactions (Siddiquie, Khan, Divakaran, & Sawhney, ). The major reason for CRF not being widely used to annotate collaborative skills is the lack of large‐scale and annotated chat data from carefully controlled collaborative activities.…”

Combining Dialog Acts and Skill Modeling: What Chat Interactions Enhance Learning Rates During AI-Supported Peer Tutoring?

“…The three categories (i.e., minimal, facilitative, constructive contribution) are considered dialog acts in the current study. Using dialog acts to analyze tutoring dialog has been employed in many previous works [55,10,36] for revealing the effectiveness of dialog tutoring and investigating student learning performance. These works developed a coding scheme to annotate dialog acts by tutors and students manually.…”

“…However, manually annotating dialog for many tutoring utterances is timeconsuming and cost-demanding [37]. To address this issue, many empirical studies [55,56,36,37,38] annotated a cer-tain amount of tutoring utterances and then employed supervised machine learning models to automate the annotating process. For example, Rus et al [55] employed a conditional random field model to train on 500 Algebra tutorial sessions.…”

“…Similarly, past work has analyzed dialog during peer tutoring and delineated effective and ineffective collaboration dynamics to develop support systems for peer tutors, which have been shown to improve their learning significantly [63]. However, these prior works on effective dialog acts and processes primarily hand-coded dialog acts [11,61,63,55] with the resulting sample sizes being too small to model learning [14]. Yet, the systems that support adaptive problem-solving during peer tutoring (e.g., intelligent tutoring systems; ITS) can record learning processes on a fine-grained level.…”

“…The present study leverages advancements in classifying dialog acts from human-to-human tutorial dialog to account for differences in learning rates during problem-solving practice of linear equations in middle school peer tutoring. Prior work creating classifiers for dialog acts from text has used data sets from higher education (e.g., [17,54]) or commercial tutoring contexts [55,36]. Therefore, it is an open question whether peer tutoring chat messages from middle school students, which are expected to be shorter and less rich in linguistic features, can be reliably classified similarly.…”