Reconstructing Neurath’s Ship: A Case Study in Reevaluating Equity in a Program of Assessment

Abstract

Inequity in assessment has been described as a “wicked problem”—an issue with complex roots, inherent tensions, and unclear solutions. To address inequity, health professions educators must critically examine their implicit understandings of truth and knowledge (i.e., their epistemologies) with regard to educational assessment before jumping to solutions. The authors use the analogy of a ship (program of assessment) sailing on different seas (epistemologies) to describe their journey in seeking to improve equity in assessment. Should the education community repair the ship of assessment while sailing or should the ship be scrapped and built anew? The authors share a case study of a well-developed internal medicine residency program of assessment and describe efforts to evaluate and enable equity using various epistemological lenses. They first used a postpositivist lens to evaluate if the systems and strategies aligned with best practices, but found they did not capture important nuances of what equitable assessment entails. Next, they used a constructivist approach to improve stakeholder engagement, but found they still failed to question the inequitable assumptions inherent to their systems and strategies. Finally, they describe a shift to critical epistemologies, seeking to understand who experiences inequity and harm to dismantle inequitable systems and create better ones. The authors describe how each unique sea promoted different adaptations to their ship, and challenge programs to sail through new epistemological waters as a starting point for making their own ships more equitable.

Education Scholarship Assessment Reconsidered: Expansion of Glassick’s Criteria to Incorporate Health Equity

“…Approaches to education scholarship are shaped by the diverse perspectives of health professions educators, which further impact approaches to health equity. 32 As such, it is important to consider how worldviews could influence the use of a health equity criterion ways in which the criterion may impact inequities and possible shortcomings specific to 3 education scholarship paradigms: postpositivist, constructivist, and critical.…”

“…However, a limitation exists if health equity is addressed, but not done so comprehensively, as it may provide a false sense of accomplishment. 32 For example, a curriculum may engage diverse and underrepresented learners, checking a health equity box as a standard for curricular rigor without addressing the many other components of health equity in curricular scholarship, such as the impact of curricular assessment or evaluation on the population of educators or learners and the potential for unintended consequences to other populations. To prevent this oversimplification of the criterion, Appendix 1 aims to provide guidance on how to apply a health equity criterion to education scholarship, and each of the 6 existing criteria have been strengthened using a health equity approach (see Table 1).…”

“…A constructivist approach considers health equity assessment as socially cocreated, locally situated, and continually redefined. 32 In this sense, the dynamic nature of the health equity criterion, which allows it to be adapted to different types of scholarship (as seen in Appendix 1), is well suited to the constructivist approach. Local institutions and organizations may engage interested parties to adapt this criterion to fit the needs of their scholarship communities.…”

Josiah Macy Jr. Foundation Conference on Ensuring Fairness in Medical Education Assessment: Conference Recommendations Report

“…In “Reconstructing Neurath’s Ship: A Case Study in Re-evaluating Equity in a Program of Assessment,” 9 the authors describe efforts to evaluate and enable equity in assessment within the internal medicine residency program at the University of Cincinnati. They consider equity in assessment through 3 different epistemological lenses: postpositivist, constructivist, and critical.…”