Detecting Disruptive Talk in Student Chat-Based Discussion within Collaborative Game-Based Learning Environments

Park, Kyungjin; Sohn, Hyunwoo; Mott, Bradford W.; Min, Wookhee; Saleh, Asmalina; Glazewski, Krista; Hmelo‐Silver, Cindy E.; Lester, James C.

doi:10.1145/3448139.3448178

Cited by 13 publications

(10 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Gamification is becoming a popular and innovative way to support both instructor and learner in the educational field with learning or behavioral challenges. Currently, [122,123] studied gamified e-quiz exercises in a formative assessment context. In this study, four different types of learning engagement (behavioral, emotional, cognitive, and agentic) are discussed for performing the assessment.…”

Section: Impact Of Assessment and Feedback On Students' Outcomesmentioning

confidence: 99%

Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education

Hooda

Rana

Dahiya

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

The core focus of this review is to show how immediate and valid feedback, qualitative assessment influence enhances students learning in a higher education environment. With the rising trend of online education especially in this COVID-19 pandemic, the role of assessment and feedback also changes. Earlier the assessment part is not considered the main focus in learning and teaching in HEIs, but now with the increase in online education, it is observed that the paradigm is shifted toward assessing those activities of students that enhance their learning outcomes. A lot of research work has been done on developing assessment strategies and techniques that can support learning and teaching effectively. Yet, there is limited research that looks at how methods applied in learning analytics can be used and possibly constitutes the assessment process. The objective of this work is to provide an exploratory and comparative study of how assessment and feedback practices can enhance students learning outcomes using AI. The key contribution of this study attempts to capture an outline of the most used artificial intelligence and machine learning algorithms for student success. The results showed that I-FCN performed better than other techniques (ANN, XG Boost, SVM, Random Forest, and Decision Trees) in all measured performance metrics. Also, the result of the comparative analysis study will help the educators, instructors, and administrators on how they could take the advantage of a data-driven approach, design less pressurized, more valid, reliable, constructive assessment findings, and connect the power of assessment and feedback to enhance the learning outcomes.

show abstract

Section: Impact Of Assessment and Feedback On Students' Outcomesmentioning

confidence: 99%

Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education

Hooda

Rana

Dahiya

et al. 2022

Mathematical Problems in Engineering

View full text Add to dashboard Cite

show abstract

“…Students' conversations can create disruption in collaborative learning environments, impeding collaborative learning processes. Recent research on bullying, off-task behavior, and disruptive talk in collaborative learning environments examined a range of word embedding techniques as well as a variety of classical machine learning and deep learning techniques (Carpenter et al, 2020;Nikiforos et al, 2020;Park et al, 2021). Nikiforos et al (2020) explored the automatic detection of aggressive behavior (i.e., bullying) in two K-12 computer-supported collaborative learning environments.…”

Section: Related Workmentioning

confidence: 99%

“…The empirical evaluations indicated that the LSTMbased off-task behavior detection model with BERT embeddings outperformed other baseline approaches. Park et al (2021) presented an LSTMbased disruptive talk detection framework in a multi-party dialogue dataset from a collaborative game-based learning environment, utilizing features from chat messages, a range of linguistic features, gender, and pre-test scores. While this work has the potential to improve learning experiences by detecting disruptions within collaborative learning settings, they disregard the unique characteristics of multi-party dialogues.…”

Section: Related Workmentioning

confidence: 99%

“…Previous work on detecting talk that can cause negative socio-emotional engagement (e.g., offtask behavior, bullying, disruptive talk) in collaborative learning environments investigated computational approaches using language models ranging from classic approaches (e.g., n-grams) and word embedding approaches (e.g., BERT). These language models have been combined with classic techniques (e.g., logistic regression, random forest) and deep learning techniques (e.g., longshort term memory networks) (Carpenter et al, 2020;Nikiforos et al, 2020;Park et al, 2021). However, the previous work either makes utterance-by-utterance predictions without taking Disruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network context into account or treats the entire multi-party conversation sequence as a continuous dialogue flow, despite the potential presence of multiple concurrent message threads with different topics.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Disruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network

Park,

Sohn,

Min

et al. 2022

Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue

View full text Add to dashboard Cite

Accurate detection and appropriate handling of disruptive talk in multi-party dialogue is essential for users to achieve shared goals. In collaborative game-based learning environments, detecting and attending to disruptive talk holds significant potential since it can cause distraction and produce negative learning experiences for students. We present a novel attention-based user-aware neural architecture for disruptive talk detection that uses a sequence dropout-based regularization mechanism. The disruptive talk detection models are evaluated with multi-party dialogue collected from 72 middle school students who interacted with a collaborative game-based learning environment. Our proposed disruptive talk detection model significantly outperforms competitive baseline approaches and shows significant potential for helping to support effective collaborative learning experiences.

show abstract

“…It is noteworthy that while CollabAssist currently provides feedback to reduce micromanagement, it is a robust and model-independent system that can provide feedback on other forms of collaborative behaviors. For example, future researchers can explore using CollabAssist to reduce off task behavior [6], bullying [55], disruptive talk [58], disputational talk [82] or exclusive language [83] during collaboration.…”

Section: Qualitative Evaluationmentioning

confidence: 99%

Improving Communication and Collaboration Using Artificial Intelligence: An NLP-Enabled Pair Programming Collaborative-ITS Case Study

Ubani

View full text Add to dashboard Cite

This dissertation investigates computational models and methods to improve collaboration skills among students. The study targets pair programming, a popular collaborative learning practice in computer science education. This research led to the first machine learning models capable of detecting micromanagement, exclusive language, and other types of collaborative talk during pair programming. The investigation of computational models led to a novel method for adapting pretrained language models by first training them with a multi-task learning objective. I performed computational linguistic analysis of the types of interactions commonly seen in pair programming and obtained computationally tractable features to classify collaborative talk. In addition, I evaluated a novel metric utilized in evaluating the models in this dissertation. This metric is applicable in the areas of affective systems, formative feedback systems and the broader field of computer science. Lastly, I present a computational method, CollabAssist, for providing real-time feedback to improve collaboration. The empirical evaluation of CollabAssist demonstrated a statistically significant reduction in micromanagement during pair programming. Overall, this dissertation contributes to the development of better collaborative learning practices and facilitates greater student learning gains thereby improving students' computer science skills.

show abstract

Detecting Disruptive Talk in Student Chat-Based Discussion within Collaborative Game-Based Learning Environments

Cited by 13 publications

References 38 publications

Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education

Artificial Intelligence for Assessment and Feedback to Enhance Student Success in Higher Education

Disruptive Talk Detection in Multi-Party Dialogue within Collaborative Learning Environments with a Regularized User-Aware Network

Improving Communication and Collaboration Using Artificial Intelligence: An NLP-Enabled Pair Programming Collaborative-ITS Case Study

Contact Info

Product

Resources

About