CPS‐Rater: Automated Sequential Annotation for Conversations in Collaborative Problem‐Solving Activities

Hao, Jiangang; Chen, Lei; Flor, Michael; Liu, Lei; Davier, Alina A. von

doi:10.1002/ets2.12184

Cited by 22 publications

(19 citation statements)

References 17 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In future studies, we plan to automate the coding of verbal indicators. Researchers have begun investigating best practices for automated assessment using the PISA and ICAP frameworks (e.g., Hao, Chen, Flor, Liu, & von Davier, 2017;von Davier et al, 2017). Such automated coding could also be used as the basis to provide timely feedback to support students and teachers (Awwal, Scoular, & Alom, 2017).…”

Section: Limitations and Future Workmentioning

confidence: 99%

Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments

Emara

Hutchins

Grover

et al. 2021

JLA

View full text Add to dashboard Cite

The integration of computational modelling in science classrooms provides a unique opportunity to promote key 21st century skills including computational thinking (CT) and collaboration. The open-ended, problem-solving nature of the task requires groups to grapple with the combination of two domains (science and computing) as they collaboratively construct computational models. While this approach has produced significant learning gains for students in both science and CT in K–12 settings, the collaborative learning processes students use, including learner regulation, are not well understood. In this paper, we present a systematic analysis framework that combines natural language processing (NLP) of collaborative dialogue, log file analyses of students’ model-building actions, and final model scores. This analysis is used to better understand students’ regulation of collaborative problem solving (CPS) processes over a series of computational modelling tasks of varying complexity. The results suggest that the computational modelling challenges afford opportunities for students to a) explore resource-intensive processes, such as trial and error, to more systematic processes, such as debugging model errors by leveraging data tools, and b) learn from each other using socially shared regulation (SSR) and productive collaboration. The use of such SSR processes correlated positively with their model-building scores. Our paper aims to advance our understanding of collaborative, computational modelling in K–12 science to better inform classroom applications.

show abstract

Section: Limitations and Future Workmentioning

confidence: 99%

Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments

Emara

Hutchins

Grover

et al. 2021

JLA

View full text Add to dashboard Cite

show abstract

“…There are numerous theory-based rubrics for coding chat or other aspects of small-group interactions (e.g., Adams et al, 2015;Andrews-Todd & Forsyth, 2020;Liu, Hao, von Davier, Kyllonen, & Zapata-Rivera, 2015;Hao, Chen, Flor, Liu, & von Davier, 2017). In many cases, though certainly not all, these codes are based on a single turn in a sequence of actions, i.e., they are defined in reference to one action performed by one group member.…”

Section: Group Processes Versus Code and Countmentioning

confidence: 99%

Measuring Dynamical Interdependence in Small-Group Collaborations

Halpin¹

2021

JLA

View full text Add to dashboard Cite

This paper addresses dynamical interdependence among the actions of group members. I assume that the actions of each member can be represented as nodes of a dynamical network and then collect the nodes into disjoint subsets (components) representing the individual group members. Interdependence among group members’ actions can then be defined with reference to a K-partite network, in which the partitions correspond to the group member components. Independence among group members’ actions can be defined with reference to a network in which the group member components are disconnected from one another. The degree to which the interactions of actual groups correspond to either of these theoretical network structures can be characterized using modified versions of existing network statistics. Taking this approach, I propose a number of network-based measures of dynamical interdependence, discuss the interpretation of the proposed measures, and consider how to assess their reliability and validity. These ideas are illustrated using an example in which dyads collaborated via online chat to complete a grade 12 level mathematics assessment.

show abstract

“…It is worth noting that we consider only the most immediate turns of discourses and ignore longer range dependency, though the extension to longer range dependency is straightforward. The reason for doing this is that the majority of short online conversations do not display long range dependency (some empirical evidence of this can be found in Hao et al, 2017a). The elements of a CTP are defined as follows,…”

Section: Conditional Transition Profilementioning

confidence: 99%

“…For example, a number of coding frameworks have been developed to analyze different aspects of the communications among team members, such as the coding framework for collaborative problem solving (CPS) skills (Liu et al, 2015), for the interactive patterns in collaboration (Andrews et al, 2017), for cohesion and language (Graesser et al, 2004; Dowell et al, 2016), and for dialog acts (Allen and Core, 1997). Based on human-coded discourse, natural language processing (NLP) techniques can be employed to automate the annotation to an accuracy level that is close to human coding (Rosé et al, 2008; Rus et al, 2015; Flor et al, 2016; Hao et al, 2017a).…”

Section: Introductionmentioning

confidence: 99%

Characterizing Interactive Communications in Computer-Supported Collaborative Problem-Solving Tasks: A Conditional Transition Profile Approach

Hao

Mislevy

2019

Front. Psychol.

Self Cite

View full text Add to dashboard Cite

Communication in a collaborative problem-solving activity plays a pivotal role in the success of the collaboration in both academia and the workplace. Computer-supported collaboration makes it possible to collect large-scale communication data to investigate the process at a finer granularity. In this paper, we introduce a conditional transition profile (CTP) to characterize aspects of each team member's communication. Based on the data from a large-scale empirical study, we found that participants in the same team tend to show similar CTP compared to participants from different teams. We also found that team members who showed more “negotiation” after the partner “shared” information tended to show more improvement after the collaboration while those who continued sharing ideas while their partners were negotiating tended to improve less.

show abstract

CPS‐Rater: Automated Sequential Annotation for Conversations in Collaborative Problem‐Solving Activities

Cited by 22 publications

References 17 publications

Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments

Examining Student Regulation of Collaborative, Computational, Problem-Solving Processes in Open-Ended Learning Environments

Measuring Dynamical Interdependence in Small-Group Collaborations

Characterizing Interactive Communications in Computer-Supported Collaborative Problem-Solving Tasks: A Conditional Transition Profile Approach

Contact Info

Product

Resources

About