Selecting performance assessment methods for experienced physicians

Hays, Richard; Davies, Helena; Beard, Jonathan; Caldon, L J M; Farmer, Elizabeth; Finucane, Paul; McCrorie, Peter; Newble, David; Schuwirth, Lambert; Sibbald, GR

doi:10.1046/j.1365-2923.2002.01307.x

Cited by 72 publications

(59 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…20 Assessment tools developed have consisted of global assessment scales as well as checklists. These have been used in multiple settings such as videotape review of patient encounters, 21 direct observation, 22 and SPs, and OSCEs. [23][24][25] Several studies have successfully looked at modifications of the American Board of Internal Medicine (ABIM) patient survey for external feedback.…”

Section: Lessons From Other Disciplinesmentioning

confidence: 99%

Assessing Interpersonal and Communication Skills in Emergency Medicine

Chan

Wallner

Swoboda

et al. 2012

Academic Emergency Medicine

View full text Add to dashboard Cite

Interpersonal and communication skills (ICS) are a key component of several competency-based schemata and key competency in the set of six Accreditation Council for Graduate Medical Education (ACGME) core competencies. With the shift toward a competency-based educational framework, the importance of robust learner assessment becomes paramount. The journal Academic Emergency Medicine (AEM) hosted a consensus conference to discuss education research in emergency medicine (EM). This article summarizes the initial preparatory research that was conducted to brief consensus conference attendees and reports the results of the consensus conference breakout session as it pertains to ICS assessment of learners. The goals of this consensus conference session were to twofold: 1) to determine the state of assessment of observable learner performance and 2) to determine a research agenda within the ICS field for medical educators. The working group identified six key recommendations for medical educators and researchers.

show abstract

Section: Lessons From Other Disciplinesmentioning

confidence: 99%

Assessing Interpersonal and Communication Skills in Emergency Medicine

Chan

Wallner

Swoboda

et al. 2012

Academic Emergency Medicine

View full text Add to dashboard Cite

show abstract

“…92 While workplace-based assessments are highly desirable for the assessment of competency, concerns have been raised about the psychometric properties of this form of assessment. 93,94 In contrast, while methods such as written examinations and oral examinations have solid psychometric properties, they are critiqued for lacking ecological validity, i.e., for not closely re-creating the practice conditions under which we want to assess competency. 95,96 Simulation-based assessments have been proposed as complementary means to assess performance and behaviour in an authentic and reliable context.…”

Section: Simulation For Assessment: State Of the Sciencementioning

confidence: 99%

Review article: Simulation in anesthesia: state of the science and looking forward

LeBlanc

2011

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

“…The final aspect of assessment to consider is that of the authenticity or validity of the assessment tasks (Hays et al, 2002, Linn et al, 1991, which is closely linked to the previous discussion regarding the nature of the rating task. There is general recognition that the most authentic assessment will be one that assesses the real task of functioning effectively as a speech pathologist in the workplace.…”

Section: Assessment and Learningmentioning

confidence: 99%

Issues in developing valid assessments of speech pathology students' performance in the workplace

McAllister¹,

Lincoln²,

Ferguson³

et al. 2009

TLCD

View full text Add to dashboard Cite

This is the peer reviewed version of the following article:McAllister, S.M., Lincoln, M., Ferguson, A. and McAllister, L. (2010). Issues in developing valid assessments of speech pathology students' performance in the workplace. Abstract BackgroundWorkplace based learning is a critical component of professional preparation in speech pathology. A validated assessment of this learning is seen to be 'the gold standard' but it is difficult to develop because of design and validation issues. These issues include the role and nature of judgement in assessment, challenges in measuring quality, and the relationship between assessment and learning. Valid assessment of workplace based performance needs to capture the development of competence over time and account for both occupation specific and generic competencies. AimsThis paper reviews important conceptual issues in the design of valid and reliable workplace based assessments of competence including assessment content, process, impact on learning, measurement issues and validation strategies. The paper then goes on to share what has been learned about quality assessment and validation of a workplace based performance assessment using competency based ratings. The outcomes of a 4 year national development and validation of an assessment tool are described. MethodsA literature review of issues in conceptualising, designing and validating workplace based assessments was conducted. Key factors to consider in the design of a new tool 2 were identified and built into the cycle of design, trialling and data analysis in the validation stages of the development process. Main contributionThis paper provides an accessible overview of factors to consider in the design and validation of workplace based assessment tools. It presents strategies used in the development and national validation of a tool COMPASS™, used in every speech pathology program in Australia, New Zealand and Singapore. The paper also describesRasch analysis, a model based statistical approach which is useful for establishing validity and reliability of assessment tools. ConclusionsThrough careful attention to conceptual and design issues in the development and trialling of workplace based assessments, it has been possible to develop the world's first valid and reliable national assessment tool for assessment of performance in speech pathology.3 What this paper addsWhat is already known on this subject: Valid and reliable assessment of workplace based learning is essential in professional entry programs but is often thwarted by difficulties in design and validation of workplace based assessment tools. These challenges have given rise to the belief that performance in the workplace cannot be validly quantified, particularly for high stakes examinations. As a result, most assessments of professional competence rely on de-contextualised assessment practices. However, it is believed that attention to conceptualisation and design issues and use of appropriate statistical analysis procedures of assessment data may o...

show abstract

Selecting performance assessment methods for experienced physicians

Cited by 72 publications

References 24 publications

Assessing Interpersonal and Communication Skills in Emergency Medicine

Assessing Interpersonal and Communication Skills in Emergency Medicine

Review article: Simulation in anesthesia: state of the science and looking forward

Issues in developing valid assessments of speech pathology students' performance in the workplace

Contact Info

Product

Resources

About