Design of an assessment system for collaborative problem solving in STEM education

Abstract

This study developed an assessment system for evaluating junior high students' collaborative problem-solving skills (CPS) in the context of science, technology, engineering, and mathematics (STEM) education. The main theoretical basis for designing the CPS learning goals of this assessment system is the matrix of collaborative problem solving proposed by the Organisation for Economic Cooperation Development (OECD). In this assessment system, there are eight assessment modules in STEM education developed by four groups of researchers who participated in this study. The modules were delivered as a web-based learning platform, where the computer was programmed to act as the agent of collaboration (rather than human-to-human collaboration), thus making this exploratory study more unique. The eight modules included problem tasks such as designing shelves, using a microwave oven, construct a house, etc. In order to ensure the validity of the assessment system, including the eight assessment modules in STEM education, three specialists were invited to examine the eight modules' content validity. Additionally, in order to ensure criterion-related validity of the effectiveness of the modules, they were tested among 222 Taiwanese junior high students. Discrimination and difficulty indices were obtained for student CPS performance on each of the eight modules. Furthermore, intercorrelations were obtained for the students' performance on each of the eight assessment modules in relation to overall and each of three collaborative problem-solving skills, i.e., (1) Establishing and maintaining shared understandings, (2) Taking appropriate action to solve the problem, and (3) Establishing and maintaining team organization. The following conclusions are drawn: (1) there was an evidence that the assessment system items had acceptable difficulty and satisfactory discrimination; (2) the intercorrelations of the students'

Direction of collaborative problem solving-based STEM learning by learning analytics approach

“…Therefore, in order to improve integrated STEM learning, it is necessary to consider social and cognitive factors when tackling STEM challenges. In this regard, collaborative problem solving (CPS) is a promising area in STEM education because of its advantages in inculcating the understanding of scientific knowledge and others' ideas, training in scientific investigation, and solving applied problems (Hesse et al, 2015;Hogan, 1999;Lin et al, 2015). Lin, Yu, Hsiao, Chang, and Chien (2018) compared the effectiveness of web-based CPS system and the classroom-based hands-on activities including the CPS scenarios from daily life, in order to develop students' CPS skills in STEM learning.…”

Technology-Enhanced Learning: An Optimal CPS Learning Application

“…In most of the CPS-related studies, the learning system was developed using hierarchical subject-based problems with default solutions and science-based content, such as physics, biology, mathematics, and chemistry [42]. In these studies, learners seldom used skills that are relevant to daily work and life [9]. Consequently, students did not obtain an opportunity to develop their CPS skills while working on day-to-day problems and problems that typically do not have predetermined solutions [43].…”

“…The first is the application's design. Four CPS experts were invited to help with the application's development and provide guidance on the problem scripts [9]. Eight problem-solving tasks were developed (fastening shelves, using a microwave, placing water, organizing a bedroom, traveling, buying a phone, building a home, and designing furniture).…”

“…A series of questions were used to assess students' CPS performance in terms of shared information flow, taking reasonable steps to solve problems, and developing and organizing a team. The overall scores of students that were obtained from the assessment of these three competences were recorded and are compared in this study [9].…”

“…With the advent of the "Fourth Industrial Revolution Age", efforts are being made to maximize the effect of learning by utilizing new technologies, such as artificial intelligence, the Internet of Things, virtual/augmented reality, and Web technologies. Recently, Web technology has developed rapidly, and online collaborative platforms have become an interface through which people can communicate and share information globally [9]. The use of online technologies for virtual collaboration is a key skill that is needed in workplaces of the future [10].…”