Theory-based metrological traceability in education: A reading measurement network

Abstract

Huge resources are invested in metrology and standards in the natural sciences, engineering, and across a wide range of commercial technologies. Significant positive returns of human, social, environmental, and economic value on these investments have been sustained for decades. Proven methods for calibrating test and survey instruments in linear units are readily available, as are data- and theory-based methods for equating those instruments to a shared unit. Using these methods, metrological traceability is obtained in a variety of commercially available elementary and secondary English and Spanish language reading education programs in the U.S., Canada, Mexico, and Australia. Given established historical patterns, widespread routine reproduction of predicted text-based and instructional effects expressed in a common language and shared frame of reference may lead to significant developments in theory and practice. Opportunities for systematic implementations of teacher-driven lean thinking and continuous quality improvement methods may be of particular interest and value.

Towards an alignment of engineering and psychometric approaches to uncertainty in measurement: Consequences for the future

“…An analogous shift has taken place in psychometrics, where recent research presented at a series of symposia jointly sponsored by several International Measurement Confederation (IMEKO) technical committees (TC-1, TC-7, and TC-13) suggests provocative new practical and theoretical correspondences between metrology and psychometrics [9][10][11][12]. In accord with those similarities, differences between the psychometric binomial model's True Score Theory (and associated Classical Test Theory) and measurement theoretical approaches to error/uncertainty [9] parallel aspects of the shift documented in the VIM and GUM.…”

“…No such guidance has yet been systematically available in psychometrics, in large part because instruments calibrated and traceable to uniform and universally available consensus unit standards are still unusual, though not unknown [10]. Routine estimation of individual measurand uncertainties is, then, encumbered by widespread reliance on True Score Theory's scaleand sample-dependent ordinal score units, and associated statistical methods.…”

“…The PRM [Polytomous Rasch Model] provides a basis for assessing the precision of measurement achieved. Theory-informed empirical research on reading ability has begun to approximate the level of detail obtained in the construction of instruments in the natural sciences [9][10][11][12]20,27,42]. Implications for uncertainty budgets and the practical interpretation of psychometric measures begin from a distinction between Type A and Type B uncertainties [5,8].…”

Avoiding AI Armageddon with metrologically-oriented psychometrics

“…In a special invited address to the international standards laboratories community (NCSLi), Leslie Pendrill, a past chair of the European Association of National Metrology Institutes, said, "The Rasch approach...is not simply a mathematical or statistical approach, but instead [is] a specifically metrological approach to human-based measurement" [9]. Experimentally tested, theoretically explained, and metrologically traceable measures [10] are still more the exception than the rule in psychology and the social sciences, but there are clear trends toward more coherent coordinations and alignment of measures across applications in education, for instance [11][12][13].…”

“…Crucially, the best approaches [16][17][18] employ a measurement approach that is on-par with physical, chemical and biological science requirements for measurement [4][5][6][7][8][9][10][11][12][13]. The measurability of a latent trait like moral reasoning is a crucial beginning to being able to cultivate it on mass scales, especially for the ethical decision making skills paramount for individuals working with AI.…”