Computer-based collaborative problem-solving assessment in Taiwan

Kuo, Bor‐Chen; Liao, Chen-Huei; Pai, Kai-Chih; Shih, Shu-Chuan; Li, Cheng‐Hsuan; Mok, Magdalena Mo Ching

doi:10.1080/01443410.2018.1549317

Cited by 15 publications

(19 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Although conversational agents have been used to assess and to facilitate collaborative interactions (e.g., [24,25,26,27]), the decision to have the students interact with computer agents during the PISA ColPS 2015 assessment was motivated by important methodological, psychometric, and logistical assessment constraints (see next section). We illustrate how agents are used in the PISA ColPS framework and also data collected in Taiwan, where the PISA framework with agents was used to assess COLPS in their country [28]. It is not possible at this time to present the ColPS items used in the 2015 assessment and to report data from PISA ColPS 2015; the data are currently being analysed and the OECD report has not been released on an assessment that involved approximately 400,000 15-year old students from three to four dozen countries.…”

Section: Problem Solving With Agents In Assessments Of Collaboratimentioning

confidence: 99%

“…The first example, the Visit , is an item that was released by OECD to the public to clarify the PISA ColPS 2015 assessment [49]. The second example was used in a study conducted in Taiwan [28] that adopted the PISA ColPS 2015 assessment framework in order to develop and test their own items.…”

Section: Scenarios To Illustrate Assessment Of Colps Skillsmentioning

confidence: 99%

“…A ColPS proficiency scale with multiple levels has not yet been determined. Meanwhile, however, these researchers [28] conducted their own assessment in Taiwan that adopted the PISA ColPS 2015 assessment framework on the internet with computer agents. Any success they had in creating and testing the materials is consistent with the view that the PISA ColPS framework is sufficiently well specified that it can be replicated by independent researchers.…”

Section: Scenarios To Illustrate Assessment Of Colps Skillsmentioning

confidence: 99%

“…In order to establish ColPS competency scales, the researchers conducted a multidimensional item response model; more specifically, they applied the multidimensional random coefficients multinomial logit model [53] with partial credit data [54]. The reliability and model fitness of the proposed ColPS assessment were all statistically acceptable [28].…”

Section: Scenarios To Illustrate Assessment Of Colps Skillsmentioning

confidence: 99%

See 3 more Smart Citations

Complex Problem Solving in Assessments of Collaborative Problem Solving

2017

Self Cite

View full text Add to dashboard Cite

Abstract:Collaborative problem solving (ColPS) proficiency was developed as a new assessment for the Programme for International Student Assessment (PISA) in the 2015 international evaluation of student skills and knowledge. The assessment framework defined by the PISA ColPS 2015 expert group crossed three major collaboration processes with four problem solving processes that were adopted from the PISA 2012 individual problem solving assessment to form a matrix of 12 specific skills. The three major collaboration processes are (1) establishing and maintaining shared understanding; (2) taking appropriate action; and (3) establishing and maintaining team organization. The four problem solving processes are exploring and understanding the problem, representing and formulating the problem, planning and executing strategies, and monitoring and reflecting on the problem-solving activities. This article discusses how the problem-solving dimension was integrated with the collaboration dimension. We also discuss how computer agents were involved in the PISA ColPS 2015 assessment in order to ensure a satisfactory assessment of collaborative problem solving. Examples of the use of agents to assess ColPS are provided in the context of a released PISA item and a project conducted in Taiwan.

show abstract

Section: Problem Solving With Agents In Assessments Of Collaboratimentioning

confidence: 99%

Section: Scenarios To Illustrate Assessment Of Colps Skillsmentioning

confidence: 99%

Section: Scenarios To Illustrate Assessment Of Colps Skillsmentioning

confidence: 99%

Section: Scenarios To Illustrate Assessment Of Colps Skillsmentioning

confidence: 99%

See 2 more Smart Citations

Complex Problem Solving in Assessments of Collaborative Problem Solving

2017

Self Cite

View full text Add to dashboard Cite

show abstract

“…In other words, the competency of "taking appropriate action to solve the problem" was challenging for Taiwanese students. Similar to Chang et al (2017), Kuo et al (2019) determined that the most difficult competency for Taiwanese students was "taking appropriate action to solve the problem". Researchers also highlighted that the competency of "establishing and maintaining shared understanding" was easier for Taiwanese students than the other two.…”

Section: Discussion Conclusion and Suggestionsmentioning

confidence: 97%

An Examination of Turkish Students' PISA 2015 Collaborative Problem-Solving Competencies

Yavuz¹,

Atar

2020

International Journal of Assessment Tools in Education

View full text Add to dashboard Cite

Technological advancements initially made it possible for individuals with different cultures from different parts of the world to consider working together, and eventually collaboration turned into a necessity. Based on this necessity, OECD measured collaborative problem-solving (CPS) competencies of 15-year-old students from the countries participating in PISA assessment in 2015. The objective of this study is to examine the data on Turkey from PISA 2015 CPS survey through Deterministic-Input, Noisy-Or-Gate (DINO) cognitive diagnostic model and to determine the students' levels of CPS competencies as well as difficulty levels of these competencies. In this context, primarily, attributes of CPS competency domain were examined and the model with the most appropriate number of attributes, classification accuracy and consistency was determined for the analysis of PISA 2015 CPS data. The sample of this descriptive study consists of 435 students. As a result of the analysis, the Q-matrix of the model data fit was observed to have the best goodness-of-fit when it had three attributes whereas model fit and classification consistency decreased as the number of attributes increased. Furthermore, the easiest competency for Turkish students was determined to be 'taking appropriate action to solve the problem' while the most difficult was 'establishing and maintaining shared understanding'. Finally, latent class distribution exposed that 56% of the students did not have any of the competencies, and 35% had all. Results of this examination indicate that all CPS competencies of Turkish students need improvement.

show abstract

Developing and validating the assessment of the skills of preservice teachers in implementing pedagogical model of collaborative problem solving

Wang

Xiuyuan

et al. 2023

Education Tech Research Dev

View full text Add to dashboard Cite

Computer-based collaborative problem-solving assessment in Taiwan

Cited by 15 publications

References 28 publications

Complex Problem Solving in Assessments of Collaborative Problem Solving

Complex Problem Solving in Assessments of Collaborative Problem Solving

An Examination of Turkish Students' PISA 2015 Collaborative Problem-Solving Competencies

Developing and validating the assessment of the skills of preservice teachers in implementing pedagogical model of collaborative problem solving

Contact Info

Product

Resources

About