A practical guide to assessing clinical decision-making skills using the key features approach

Farmer, Elizabeth; Page, Gordon

doi:10.1111/j.1365-2929.2005.02339.x

Cited by 95 publications

(70 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…On reflection, the culture of those (existing) assessment designs was not considered the most relevant in a context of a Model 2 view of practice. Instead there should be a change away from the application of knowledgebased tests, away from reliance on simulated patient OSCEs, towards problem solving written tests such as key features exams 23 and extended matching components 35 , a move away from the long case and towards the mini clinical examination (CEX) 21,22 type of examination processes, supplemented by direct observation of procedural skills (DOPS) and case based discussion. Most importantly, an assessment process should include an element of work-based assessment, as some capabilities cannot be captured unless a real situated practice is observed over time.…”

Section: Assessment Tool Choicesmentioning

confidence: 99%

See 1 more Smart Citation

Assessment of osteopaths: Developing a capability-based approach to reviewing readiness to practice

Stone¹,

Boud

Hager

2011

International Journal of Osteopathic Medicine

View full text Add to dashboard Cite

ABSTRACT.A new approach to assessment design is considered through the process of the development of a set of capabilities for osteopathic practice that start from an understanding of a concept of 'practice' applicable to real, situated osteopathic healthcare. Appropriately framed capabilities inform a variety of assessment needs, allowing different tools to assess common standards across many credentialing, high stakes, summative and formative situations, and through work based practice. An argument is made for the inclusion of a work-based phase of assessment in high stakes examinations for overseas trained candidates as the best way to capture real / situated practice enabling the assessment process to contribute to on-going professional learning. The relationship of assessment to learning is regarded as vital to the development of many aspects of regulatory policy, programme accreditation, continuing professional development and needs to be considered by stakeholders concerned with maintaining and improving standards of practice.

show abstract

Section: Assessment Tool Choicesmentioning

confidence: 99%

“…22 Even a 'clinically oriented problem solving approach' utilised within newly designed tools such as extended matching multiple choice and key features examination 23 has a narrower focus than the real-life experience of practice.…”

Section: Practice and Its Assessmentmentioning

confidence: 99%

Assessment of osteopaths: Developing a capability-based approach to reviewing readiness to practice

Stone¹,

Boud

Hager

2011

International Journal of Osteopathic Medicine

View full text Add to dashboard Cite

show abstract

“…Each encounter is scored by a single assessor with a key features checklist (Farmer & Page 2005) and a global grade with five categories (Fail, borderline, clear pass, good pass, excellent). Standard setting is undertaken using the Borderline Regression method, with a comprehensive range of whole test and station level quality analyses undertaken post-hoc (Pell and Roberts 2006;Pell et al 2010).…”

Section: Exemplar Case Studies and Contextmentioning

confidence: 99%

Managing extremes of assessor judgment within the OSCE

et al. 2016

View full text Add to dashboard Cite

Rogue or rational? Investigating the impact of the extremes of assessor judgement within the OSCE. Abstract ContextThere is a growing body of research investigating assessor judgements in complex performance environments such as OSCE examinations. Post-hoc analysis can be employed to identify some elements of unwanted assessor variance. However, the impact of individual, apparently extreme O"CE pass/fail decisions has not been previously explored, This paper uses a range of studies as examples to illustrate O"CE and gives pragmatic suggestions to successfully alleviating problems. Method & ResultsWe used real OSCE assessment data from a number of examinations where at station level, a single examiner assesses student performance using a global grade and a key features checklist. Three exemplar case studies where initial post hoc analysis has indicated problematic individual assessor behaviour are considered and discussed in detail, highlighting both the impact of individual examiner behaviour and station design on subsequent judgments. ConclusionsIn complex assessment environments, institutions have a duty to maximise the defensibility, quality and validity of the assessment process. A key element of this involves critical analysis, through a range of approaches, of assessor judgments. However, care must be taken when assuming that apparent aberrant examiner behaviour is automatically just that.

show abstract

“…Both item types are widely employed for evaluating medical and clinical skills. However, results from medical licensing authorities suggest that the write-in questions are more effective in identifying and discriminating low from high performing candidates (Farmer & Page, 2005). Each year, over 12,000 candidates write the Canadian licensure examinations (Medical Council of Canada [MCC], 2014), and despite the effectiveness and widespread use of write-in CDMs in these examinations, this item type is manually scored by human markers who must first be trained for the grading task and then be monitored throughout the marking process to ensure that their scores are reliability produced.…”

Section: Introductionmentioning

confidence: 99%

Using Automated Scoring to Evaluate Written Responses in English and French on a High-Stakes Clinical Competency Examination

et al. 2015

View full text Add to dashboard Cite

We present a framework for technology-enhanced scoring of bilingual clinical decision-making (CDM) questions using an open-source scoring technology and evaluate the strength of the proposed framework using operational data from the Medical Council of Canada Qualifying Examination. Candidates' responses from six write-in CDM questions were used to develop a three-stage-automated scoring framework. In Stage 1, the linguistic features from CDM responses were extracted. In Stage 2, supervised

show abstract

A practical guide to assessing clinical decision-making skills using the key features approach

Abstract: The KFP format provides educators with a flexible approach to testing clinical decision-making skills with demonstrated validity and reliability when constructed according to the guidelines provided.

Cited by 95 publications

References 5 publications

Assessment of osteopaths: Developing a capability-based approach to reviewing readiness to practice

Assessment of osteopaths: Developing a capability-based approach to reviewing readiness to practice

Managing extremes of assessor judgment within the OSCE

Using Automated Scoring to Evaluate Written Responses in English and French on a High-Stakes Clinical Competency Examination

Contact Info

Product

Resources

About