Evaluating pharmacotherapy education: urgent need for hard outcomes

Abstract

Recently, a call was made for more high-quality articles on clinical pharmacology and therapeutics (CPT) education [1]. Not only is there a lack of research in this field, but also the quality of the available literature is disturbing. Most of the evaluation methods currently used in undergraduate CPT education rely too heavily on 'soft' endpoints, such as satisfaction with the education received, and on intermediate endpoints, such as increase in knowledge and skills instead of improvement of patient care. Moreover, measures of knowledge and skills are often not valid, because selfassessment does not reflect actual knowledge and skills. Both issues are discussed in this letter.First, the lack of 'hard endpoints' is a problem. In educational research, it is common to classify endpoints on the basis of the four levels of evaluation described in Kirkpatrick's model of impact of education (modified by Freeth) [2,3] (Table 1). Unfortunately, studies of CPT education with hard endpoints (levels 3 & 4) are rare. For example, in four systematic reviews of undergraduate CPT education [4][5][6][7], only 2 of the 63 included articles (3%) used level 3 or 4 endpoints to evaluate their educational interventions (Table 1). In fact, most of the included articles (47; 75%) used level 2 endpoints, so-called intermediate endpoints. Improvement in intermediate endpoints (Kirkpatrick level 2) does not necessarily translate into improvement in patient care (Kirkpatrick level 4). Thus it is difficult to interpret the results of level 2 studies in terms of their value for CPT education.Secondly, the validity of the (soft) endpoints used in educational research is sometimes dubious. A recent study showed that self-reported confidence in prescribing skills is poorly correlated with assessed competence [8]. A logical consequence of this finding is that self-reported confidence should not be used as the primary measure of students' prescribing skills at the undergraduate level. This prompted us to question whether the same is true for pharmacology and pharmacotherapy knowledge. Is self-evaluated knowledge also poorly correlated with objectively assessed knowledge? To evaluate this hypothesis, we reanalysed data obtained from bachelor and masters students in an earlier study by Keijsers et al. [9]. In this study, students' knowledge of basic pharmacology and clinical pharmacology was assessed using a written multiple choice assessment, and students rated their confidence in applying pharmacology and pharmacotherapy knowledge in clinical practice. Correlations between self-reported confidence and assessed basic or applied knowledge were absent or weak, with Pearson's correlations ranging from r = 0.07 to r = 0.25 (P < 0.05). This mismatch between confidence scores and knowledge scores has also been reported for residents [10].A commonly used approach to bridge the gap between knowledge/skills assessments and clinical practice is simulation-based education (Kirkpatrick level 3). While simulation has been shown to improve important learning ...

Mandatory medication content in the curricula of six health care personnel types with patient contact in Denmark

“…European clinical pharmacology educators have called for improved curricula in order to ensure safe and effective prescribing . One way to improve knowledge about prescribing for both physicians and nurses includes clinical rotations or student‐run clinics, where students can gain experience with real‐life patients . Others have pointed out that pharmacists are an underutilised source of support for the prescriber …”

Student Evaluation of a Learning Enhancement in Anatomy Program (LEAP) during the COVID-19 Pandemic: A Retrospective Study

“…Potential mismatches in Kirkpatrick levels have been expounded in some studies. 20,21 Proposed reasons for mismatch include a lack of validated tools to evaluate knowledge domains more accurately. Future studies can further explore the relationship between Levels 1 and 2 for anatomy and histology in each anatomical region/organ system.…”

An Interprofessional Student‐Run Medication Review Program: The Clinical STOPP/START‐Based Outcomes of a Controlled Clinical Trial in a Geriatric Outpatient Clinic

“…Unlike most studies on interprofessional education (IPE), we demonstrated the importance of IPE for clinical STOPP/START-based outcomes, which is part of the World Health Organization's IPE definition. 38,39 Because this intervention study was successful, we now review the medication of all patients attending the memory clinic, and other healthcare professionals are enquiring about the possibility of implementing a student-run medication review team in their outpatient clinics.…”