AMEE Guide No. 123 – How to read studies of educational costs

Abstract

Healthcare and health professions education share many of the same problems in decision making. In both cases, there is a finite amount of resources, and so choices need to be made between alternatives. To navigate the options available requires effective decision making. Choosing one option requires consideration of its opportunity cost -the benefit forgone of the other competing options.The purpose of this guide is to introduce educational decision-makers to the economic concept of cost, and how to read studies about educational costs to inform effective cost-conscious decision-making.This guide leads with a brief review of study designs commonly utilised in this field of research, followed by an overview of how study findings are commonly presented. The tutorial will then offer a four-step model for appraising and considering the results of an economic evaluation. It asks the questions: 1) Can I trust the results? 2) What are the results telling me? 3) Could the results be transferred to my context? 4) Should I change my practice?Educational decision-makers are uniquely positioned to create change in teaching and learning practices. Data published from economic evaluations can be a powerful decision-making aide. As the number of studies that examine the cost and value of health professions education grows, education decision-makers will require increasing skill in understanding, appraising, and considering the study findings, to ensure that educational activities achieve optimal value for a given spend. PRACTICE POINTS1. The main reason to consider cost and value in health professions education is to ensure that teaching and assessment deliver optional educational value for a given spend.5. Appraising a study's reporting of costs can be assisted by asking 1) How were the interventions cost-items identified? 2) how were the cost items measured? And 3) how were the cost items priced?6. Uncertainty around point estimates of cost and value is evaluated through the sensitivity analysis, which considers the impact of varying assumptions and conditions. Decision-makers should consider the sensitivity scenario which best aligns with their own context, to better interpret study transferability.7. Value is highly context dependent, and decisions concerning the uptake or rejection of an educational innovation are ultimately affected by factors and forces beyond effectiveness and costs.

COVID 19: Designing and conducting an online mini-multiple interview (MMI) in a dynamic landscape

“…This will allow us to evaluate if the adapted MMI has comparable predictive validity to our "usual" (non-COVID-19) MMI, and MMIs reported in the literature. Additionally, there has been increasing focus on the cost-effectiveness of medical education generally (Maloney et al 2019), selection into medical school specifically (Schreurs et al 2018). Taking the view that every cloud has a silver lining, we have the opportunity for a natural cost-benefit analysis.…”

Cost evaluations in health professions education: a systematic review of methods and reporting quality

“…Knowledge of costs can be used to improve education programmes, firstly from a summative view in terms of the identification of which is a more cost‐effective option between programmes, but also from a formative view in the identification of cost drivers within a programme . Although value matters more than cost alone, judgements of value are highly context‐dependent and will depend on the perspective of the decision maker and the objectives of the educational intervention . Therefore, accurate and full evaluations of costs, reported in a complete manner, are necessary for cost evaluations to inform decision making.…”

These may not be the courses you are seeking: a systematic review of open online courses in health professions education

“…While two articles included information related to the expense of course development (50,000 Euros and 10,000-50, 000 dollars) [28,46], no studies reported outcome measures relating to either a simple cost or value analysis, or comparative costs in the form of cost-benefit or cost-effectiveness analysis. We looked for evidence across the full spectrum of cost and value analyses, including cost-analyses (where outcomes are not considered), and breakeven analyses, and comparative approaches such as cost-minimisation analyses (where the outcomes are assumed equal), cost-benefit analyses (where costs and effects are considered in monetary units), and cost-effectiveness analyses (where outcomes are retained in natural units, such as measures of learning) [67,68].…”