Naturalistic observations of metacognition in engineering: Using observational methods to study metacognitive engagement in engineering

Abstract

Background The use of metacognition is critical to learning, especially in fields such as engineering that involve problem‐solving and difficult conceptual material. Due to limitations with current methodological approaches, new methods are needed to investigate engineering students' metacognitive engagement in learning situations that are self‐directed, such as study groups. Purpose Our purpose was to develop an approach to investigate the metacognitive engagement of undergraduate engineering students in self‐directed learning environments. The Naturalistic Observations of Metacognition in Engineering (NOME) Observational Protocol and Coding Strategy is a qualitative data collection method that allows researchers to observe the behaviors of students who are studying in groups to determine the student's engagement in different metacognitive practices. The NOME is intended to be used by researchers interested in studying online metacognitive behaviors without the direct interference of a methodological approach. Design/Method We observed three study groups where students were working on an engineering problem‐solving homework assignment. Using a taxonomic definition of metacognition, we coded episodes of observation transcripts to identify behaviors that represented key definitions in the taxonomy. Results We combined subcodes and descriptions of behaviors with key definitions to develop a coding strategy useful for future observational studies. Evidence of intercoder agreement and agreement in unitizing indicates that the coding strategy can reliably be used by multiple trained coders to identify metacognitive engagement. Conclusions The reliability evidence shows that the NOME may be a useful tool for researchers in engineering education interested in studying the metacognitive habits of engineering students in self‐directed study.

Uncovering Epistemologies and Values of Our Qualitative Engineering Education Research Community: Listening for Voices

“…Epistemological unconsciousness also appeared in the couched studies that were somewhat common in articles we analyzed McCord & Matusovich, 2019;Mobley et al, 2019). These couched studies were essentially studies that analyzed a small number of participants, which were derived from larger datasets.…”

Board 133: "This Seems Reasonable": Using Epistemic Cognition and Metacognition to Justify the Reasonableness of Solutions in Senior Design

“…For the observation portion of this study, we will be using an observational coding strategy used in a previous study [32]. In the original data collection, groups of engineering students who had formed study groups were video and audio recorded.…”

Developing the ESLS - Engineering Students Learning Strategies Instrument

“…In order to help students, there is a need to understand both the learning strategies that they are currently using and what they should ideally be doing. Recent research that does exist on engineering students' learning strategies tends to focus in the cognitive factors (e.g., (McCord & Matusovich, 2019), Litzinger et al, 2010; or tends to look at perceptions of an beliefs about problems solving (e.g., Kirn & Benson, 2018;McNeill et al, 2016). However, neither approach has yet yielded an instrument that faculty can use to measure student learning in a practical, user-friendly way which is the goal of our research.…”