Alternative model for administration and analysis of research-based assessments

Abstract

Research-based assessments represent a valuable tool for both instructors and researchers interested in improving undergraduate physics education. However, the historical model for disseminating and propagating conceptual and attitudinal assessments developed by the physics education research (PER) community has not resulted in widespread adoption of these assessments within the broader community of physics instructors. Within this historical model, assessment developers create high quality, validated assessments, make them available for a wide range of instructors to use, and provide minimal (if any) support to assist with administration or analysis of the results. Here, we present and discuss an alternative model for assessment dissemination, which is characterized by centralized data collection and analysis. This model provides a greater degree of support for both researchers and instructors in order to more explicitly support adoption of research-based assessments. Specifically, we describe our experiences developing a centralized, automated system for an attitudinal assessment we previously created to examine students' epistemologies and expectations about experimental physics. This system provides a proof of concept that we use to discuss the advantages associated with centralized administration and data collection for research-based assessments in PER. We also discuss the challenges that we encountered while developing, maintaining, and automating this system. Ultimately, we argue that centralized administration and data collection for standardized assessments is a viable and potentially advantageous alternative to the default model characterized by decentralized administration and analysis. Moreover, with the help of online administration and automation, this model can support the long-term sustainability of centralized assessment systems.

Investigating and promoting epistemological sophistication in quantum physics

“…Although there are notable exceptions, including studies of students' epistemologies and expectations of experimental physics (e.g. [6]). In this study, we explore domain-specific epistemological commitments within physics, seeking to add to discussions around when and where we might treat physics as monolithic and where we consider sub-domains.…”

Introductory physics labs: We can do better

“…3 Labs characterized as "guided" as opposed to open-ended also showed similar negative effects. 4 Looking at student thinking To try to understand why those lab courses were failing to meet their intended aims and what might be done to make them more effective, we turned to the basic method underlying most physics education research: looking at student thinking. We explored that mental terrain through an extended set of interviews with focus groups with 32 students who carried out undergraduate research over a summer semester; all the students had already taken the introductory physics lab sequence.…”

In-class vs. Online Administration of Concept Inventories and Attitudinal Assessments

“…With these changes, one instructor had an online post-test participation rate that exceeded that of the paper post-test participation rate of every other instructor. Given the growing focus on centralized, online administration of research-based assessments [10], our study offers hope that, when scaffolded properly, online assessment participation rates can rival those of in-class assessments.…”