This article focuses on “measurement for improvement,” which is the analytic work critical to making and spreading effective changes in quality improvement approaches to system transformation. Quality improvement methods aim to trigger and accelerate the learning of people within a system to make that system work better. This is accomplished through the careful attention of those who experience and enact the system at various levels and by leveraging the expertise from within and outside of the system. Measurement provides critical fuel for that learning. “Measurement for improvement” serves as an umbrella term encompassing a range of measures discussed by quality improvement scholars. These other terms include “process measures,” “practical measures,” and “pragmatic measures.” Data about system outcomes are often a key motivating factor for spurring improvement efforts, for example, low students’ graduation rates or low achievement test scores. However, outcome measures typically fall short of the data needed to inform the day-to-day work of trying changes in practice and learning from these change efforts. Instead, measures of key processes that lead to system outcomes are needed. Measures for improvement need to be closely connected to key processes, timely, and easy to collect and analyze on a regular basis by people in the system. They must also function within social processes that engender trust and transparency so that improvers can learn from failures as well as successes. The education sector has looked to quality improvement efforts in health care as a model for this work, and this article draws heavily on key texts from quality improvement in health care. However, there are some key differences in the types of data regularly available between the fields of health care and education; these differences prompt attention to certain measurement concerns, which are taken up in the references included in this article. The article begins with History and Lineage, which includes some key references that trace quality improvement ideas from industry to health care to education. The next section, Features of Measures for Improvement covers how the function of measures in a quality improvement endeavor shapes the form they take. The next section, A Set of Measures to Inform Improvement, discusses the types of measures needed in combination to inform quality improvement work in systems. The following section, Rigor of Measures to Inform Improvement, addresses what it means to have rigorous measures for improvement, in the context of the educational field where the phenomena of interest are often difficult to see or count. Another challenge of measurement for improvement is taken up in the next section, Analytic Infrastructure for Measurement for Improvement. The challenges of data collection, analysis, and consumption within the busy work lives of improvers imposes constraints and considerations to the social and technological infrastructure that enables the use of measures for improvement purposes. Finally, this article concludes with cases that illustrate measurement for improvement in educational contexts.
Despite the ease of accessing a wide range of measures, little attention is given to validity arguments when considering whether to use the measure for a new purpose or in a different context. Making a validity argument has historically focused on the intended interpretation and use. There has been a press to consider both the intended and actual interpretations and how users make sense of the data when constructing validity arguments, but the practice is not widespread. This paper contributes to existing research on validity by highlighting the value of attending to the actual interpretation and use of a measure aimed at supporting instructional improvement in mathematics. We describe the use of the same measure across two contexts to highlight the importance of attending to characteristics of both users and the contexts in which the measures are used when assessing the validity of inferences for the purpose of instructional improvement efforts.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.