Examining the efficacy of a professional development assessment tool

Abstract

Professional development is an effective way to diffuse evidence-based instructional practices and support their implementation. At the same time, assessing the impacts of professional development opportunities in physics education is still an emerging field. In this paper, we examine the efficacy of an assessment instrument based on the Theory of Planned Behavior, a theoretical framework originating from psychology. The theory has been used in other fields to understand and predict behaviors, but it has not been utilized in STEM Education research. The theory posits that participants' attitudes, norms, and perceived behavioral control beliefs toward a particular behavior (e.g., implementation of active learning) predict intentions which, in turn, determine and explain that behavior. Previous work presented empirical data that show that a link exists between intentions (as measured by the instrument) and adoption of active learning strategies (as measured by the COPUS observation protocol). In this paper, we focus on examining evidence for the validity and reliability of measurements produced by the instrument, which is necessary to provide further validity evidence to the empirical results of the prior study. We also discuss the retrospective pre-test methodology for data collection which helps minimize the effects of biases associated with self-reported data collected via traditional pre-post survey administrations. We used Arjoon, Xu, and Lewis's framework for validity and provide evidence for the reliability, internal structure, and temporal stability of our measures. We will also compare results from our instrument to those obtained using the Approaches to Teaching Inventory.