The assessment of collaborative problem solving in PISA 2015: Can computer agents replace humans?

Herborn, Katharina; Stadler, Matthias; Mustafić, Maida; Greiff, Samuel

doi:10.1016/j.chb.2018.07.035

Cited by 51 publications

(29 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The interaction interface remained unchanged (i.e., there was a limited set of predefined choices and chat messages). Herborn et al (2018) found no CPS performance differences between the H-A and H-H conditions; that is, both the H-A and the H-H conditions loaded on one overarching CPS factor with very little method variance. However, compared with the H-A condition, students interacted more in the H-H condition.…”

Section: Comparisons Of Pisa and Atc21s Frameworkmentioning

confidence: 77%

“…A descriptive, empirical, comparative analysis of the PISA and ATC21S assessments would be worthwhile to conduct. A major step in that direction was pursued by Herborn, Stadler, Mustafić, and Greiff (2018), who compared an H-A condition with an H-H condition. In that study, the PISA framework design and PISA CPS tasks were used, but in the H-H condition, the computer-simulated agents were replaced with humans in the same classroom (i.e., an actual peer), and students were told that they were working with one of their classmates.…”

Section: Cps Theoretical Frameworkmentioning

confidence: 99%

See 1 more Smart Citation

Advancing the Science of Collaborative Problem Solving

Graesser

Fiore

Greiff

et al. 2018

Psychol Sci Public Interest

Self Cite

220

171

View full text Add to dashboard Cite

Collaborative problem solving (CPS) is an important 21st century skill that is increasingly recognized as being critical to efficiency, effectiveness, and innovation in the modern global economy (Fiore, Graesser, & Greiff, 2018; Organisation for Economic Cooperation and Development [OECD], 2017a). CPS has attracted interest in international assessments; national assessments of middle and high school students; and training in colleges, industry, and the military (Care,

show abstract

Section: Comparisons Of Pisa and Atc21s Frameworkmentioning

confidence: 77%

Section: Cps Theoretical Frameworkmentioning

confidence: 99%

Advancing the Science of Collaborative Problem Solving

Graesser

Fiore

Greiff

et al. 2018

Psychol Sci Public Interest

Self Cite

220

171

View full text Add to dashboard Cite

show abstract

“…Bireylerin belirli bir alandaki performansları ile yanıtlama süresi arasındaki ilişkiyi inceleyen zengin bir alanyazın olmasına rağmen, bir maddeyi cevaplarken yapılan toplam eylem sayısının öğrenci performansını nasıl etkilediğine ilişkin sınırlı sayıda çalışmaya rastlanmıştır (He, von Davier, & Han, 2018;Herborn, Stadler, Mustafić, & Greiff, 2018 Okuduğunu anlama becerisi: Öğrencilerin PISA 2015'teki okuduğunu anlama alanında yanıtladıkları maddelerden elde ettikleri başarı puanlarıdır. Ekonomik Kalkınma ve İşbirliği Örgütü bazen de İktisadi İşbirliği ve Gelişme Teşkilatı (Organisation for Economic Co-operation and Development-OECD) (2017) tarafından okuduğunu anlama becerileri bireyin yazılı metinleri kullanarak, üzerinde düşünerek, anlayarak amaçlarını gerçekleştirme, bilgisini ve potansiyelini geliştirme ve toplum içerisinde katılımına yönelik beceriler olarak tanımlamaktadır.…”

Section: Bilgisayar Ortamında Kaydedilen Madde Yanıtlama Verilerinin unclassified

“…Thus, it was expected that students having a lower ability on ICT would spend more time on items, and it was assumed that the interaction between ICT competence and time would negatively affect overall student performance. Although extensive research has been carried out on the relationship between RT and test-takers' ability, a limited number of research (He, von Davier, & Han, 2018;Herborn, Stadler, Mustafić & Greiff, 2018) was found in the literature regarding how the number of actions taken to solve a given item affect student performance. Since these studies were in the context of problem-solving behaviors, additional research can be undertaken to find associations between the number of actions taken by students while answering items during testing and students' overall performance.…”

mentioning

confidence: 99%

The Effects of Log Data on Students’ Performance

Yavuz

2019

Eğitimde Ve Psikolojide Ölçme Ve Değerlendirme Dergisi

View full text Add to dashboard Cite

This study aimed to assess the relationships between response times (RTs), the number of actions taken to solve a given item, and student performance. In addition, the interaction between the students' information and communications technology (ICT) competency, reading literacy, and log data (time and number of actions) were examined in order to gain additional insights regarding the relations between student performance and log data. The sample consisted of 2 348 students who participated in the triennial international large-scale assessment of the Programme for International Student Assessment (PISA). For the current study, 18 items in the one cluster of the 91st booklet were chosen. To achieve the aim of the study, explanatory item response modeling (EIRM) framework based on generalized linear mixed modeling (GLMM) was used. The results of this study showed that students who spent more time on items and those that took more actions on items were more likely to answer the items correctly. However, this effect did not have variability across items and students. Moreover, the interaction only with reading and the number of actions was found to have a positive effect on the students' overall performance.

show abstract

“…That is, while participating in collaborative work that led to cooperation and competitive behavior, the students analyzed the contribution that others made to problem-solving [17]. These findings led scientists to use computer conversational agents to train students in a range of educational fields to solve competitive behavior problems, such as developing an inferential CPS platform that uses computer agents who collaborate with one another [18][19][20]. With the advent of Web 2.0, the use of conversational agents such as Chatbots constitutes a new chapter in the evolution of Web technology [21].…”

Section: Introductionmentioning

confidence: 99%

Technology-Enhanced Learning: An Optimal CPS Learning Application

Chien

2019

Sustainability

View full text Add to dashboard Cite

Collaborative problem-solving (CPS) is highly valued in the sustainability of learning to foster the key soft power of talent for the future. In this study, a CPS learning application was built to train and assess individuals with the aim of increasing CPS skills. For effective learning to take place, several issues need to be carefully considered, and these were investigated while testing the proposed application. This study examined the impact of collaborative interactions (CIs) (human–computer agent (HCA) and human–human (HH) interactions) on the CPS performance of students. Gender and learning styles, which may have interaction effects with CIs on CPS performance, were also explored. The results show that the students’ CPS performance in HCA was significantly greater than that in HH. The interaction effect between gender and CI was not significant. The impact of learning style on CPS performance in HH was not significant. In contrast, in HCA, students with verbal, global, and reflective learning styles performed significantly better on CPS tasks than did students with visual, sequential, and active learning styles. Finally, we discussed the optimal ways to teach CPS and the practical effects of a CPS learning application.

show abstract

The assessment of collaborative problem solving in PISA 2015: Can computer agents replace humans?

Cited by 51 publications

References 13 publications

Advancing the Science of Collaborative Problem Solving

Advancing the Science of Collaborative Problem Solving

The Effects of Log Data on Students’ Performance

Technology-Enhanced Learning: An Optimal CPS Learning Application

Contact Info

Product

Resources

About