Examining the Validity of Scores From an Instrument Designed to Measure Metacognition of Problem Solving

Abstract

The authors' purpose in this study was to gather evidence on the validity of scores on an instrument designed to evaluate levels of metacognition in problem solving--Swanson's 15-item metacognition questionnaire (1990, 1993). Forty-three participants took part in this study: 21 professors and 22 graduate students from several Western and Midwestern campuses in the fields of psychology and educational psychology. A Kendall Coefficient of concordance (W) was performed to determine the consistency of ranked responses for professors, students, and both groups combined, and to determine whether their responses matched those of the scoring key for the instrument we studied. This analysis indicated problems for some of the responses to this instrument, as demonstrated through the ranking task developed in this study. Many of the professors disagreed with the ranking of the responses suggested by Swanson. There is certainly a need for more instruments to validly measure metacognition. This instrument is one that, with some modification, might be used more in research on metacognition.

Cognitive flexibility and adaptive decision‐making: Evidence from a laboratory study of expert decision makers

“…Past research has found that well‐ and ill‐structured problems require different sets of skills and processes, and therefore performance should be evaluated independently, taking into account the different problems’ objectives (Jonassen, ; Shin, Jonassen, & McGee, ; Sigler & Tallent‐Runnels, ). The objective for Task 1 was defined as “score as many points as you can,” while for Task 2 it was “keep your tribe safe.” Performance in Task 1 was measured with each participant's total cumulative score.…”

Factors affecting metacognition of undergraduate nursing students in a blended learning environment

“…Metacognition is an active process of knowing, or being acutely aware of one's cognitive state with the ability to complete a given task. Psychologists and educators recognize the need to guide students to solve complex problems and have proceeded to enhance students' abilities in this area …”

“…Psychologists and educators recognize the need to guide students to solve complex problems and have proceeded to enhance students' abilities in this area. 10 Ireland, Johnson, Adams, Eboh and Mowatt conducted an opinion survey of participants in a blended learning course in nursing and midwifery education. 11 The module was rated by all 36 participants as useful, although their opinions on the online discussion part of the module were mixed.…”

Metacognitive Inventory for nursing students in Taiwan: instrument development and testing

“…2007). Psychologists and educators recognize the need to teach students to solve complex problems and have proceeded to enhance students’ abilities in this area (Sigler & Tallent‐Runnels 2006). They further advocate that methods be developed to facilitate the solving of ill‐structured problems.…”

“…Students without adequate metacognitive skills would have difficulty following a teacher's instructions to identify relevant information or understand cause and effect relationships (Garrett et al 2007). Psychologists and educators recognize the need to teach students to solve complex problems and have proceeded to enhance students' abilities in this area (Sigler & Tallent-Runnels 2006). They further advocate that methods be developed to facilitate the solving of ill-structured problems.…”