A Theory‐Based Framework for Assessing Domainl‐Specific Problem‐Solving Ability

Abstract

Is problem solving a global construct? What components can be assessed? What multiple assessment strategies can be used?

Complex problem solving in educational contexts—Something beyond g: Concept, assessment, measurement invariance, and construct validity.

“…According to Funke and Frensch (2007), a problem solver has to overcome barriers by applying operators and tools to solve a problem. However, problem solving may take place in different educationally relevant domains, and a large body of research has been conducted in domain-specific areas such as mathematical, scientific, or technical problem solving (Sugrue, 1995). Besides these domain-specific approaches, the idea of domain-general processes generally involved in problem solving was taken up by the European line of research on complex problem 2 solving mentioned above (e.g., Dörner, 1986;Funke, 2001;Funke & Frensch, 2007).…”

Conclusion

“…These are all substantial effect sizes and mean that scaffolding leads to strong learning outcomes across the three assessment levels (Sugrue, 1995). The findings suggest that by employing computer-based scaffolding along with problem-centered instructional models, one can erase the former liability of problem-centered instructional models-poor concept learning.…”

Intended Learning Outcomes and Assessment of Computer-Based Scaffolding

“…Indeed, one is often advised to consider assessment before even designing objectives and instructional materials/strategies (Gagné, 1965;Wiggins & McTighe, 2005). By considering how scaffolding's influence on cognitive outcomes varies according to how it is assessed-at the concept, principles, or application level (Sugrue, 1995)-one can see if scaffolding as a whole delivers stronger impacts on content learning or various types of higher-order thinking skills. It is important to consider this alongside the intended learning outcome, as (a) just because an intervention is designed to increase content learning or higherorder skills does not necessarily mean that it does, and (b) just because scholars claim that scaffolding is intended to help students enhance their skill in a particular area does not always mean that the learning is being assessed at that level.…”

“…When examining assessment of learning results from the use of computer-based scaffolding, it is useful to consider the assessment framework of Sugrue (1995), who classified assessments in terms of whether they measure at the concept, principles, or application level (see Fig. 4.2).…”

“…After all, an assessment that is on the topic of problem-solving does not necessarily assess problem-solving ability. To assess problem-solving, one would need to assess students' abilities to define the problem, determine needed information, and find and synthesize the needed information to arrive at a solution Sugrue, 1995). To assess learning appropriately, it is important to consider the constructs of interest, defined as a characteristic of an individual or group (e.g., intelligence, fluency, and argumentation ability) that cannot be directly measured, and for which one can only measure certain related behaviors (e.g., ability to construct an argument given a scenario and argument construction parameters) Kerlinger & Lee, 2000;Messick, 1989).…”