Quantitative Validation of a General Competency Composite Assessment Evaluation

Reisdorff, Earl J.; Carlson, Dale J.; Reeves, M. E.; Walker, G. K.; Hayes, Oliver W.; Reynolds, Brian

doi:10.1111/j.1553-2712.2004.tb00773.x

Cited by 13 publications

(11 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Eight EM residency programs participated in a study to develop, implement, and obtain evidence to support the validity of a global rating tool. 34,35 This instrument had acceptable reliability statistics and demonstrated the progressive acquisition of the SBP competency across three years of residency training.…”

Section: Ratings and Survey-based Instrumentsmentioning

confidence: 95%

See 1 more Smart Citation

Assessing Systems‐based Practice

Chen¹,

O’Sullivan

Pfennig

et al. 2012

Academic Emergency Medicine

View full text Add to dashboard Cite

The conceptual definition of systems-based practice (SBP) does not easily translate into directly observable actions or behaviors that can be easily assessed. At the Academic Emergency Medicine consensus conference on education research in emergency medicine (EM), a breakout group presented a review of the literature on existing assessment tools for SBP, discussed the recommendations for research tool development during breakout sessions, and developed a research agenda based on this discussion.ACADEMIC EMERGENCY MEDICINE 19:1366-1371 © 2012 by the Society for Academic Emergency Medicine T he inclusion of systems-based practice (SBP) as one of the Accreditation Council for Graduate Medical Education (ACGME) core competencies highlights the importance of the health care system on the ability of a physician to provide competent and effective patient care.1 Physicians must be able to collaborate with other members of the health care team, consider costs when weighing risks and benefits, improve system performance by identifying system errors and implementing potential solutions, and continue to advocate for quality patient care. This set of skills is also described in the CanMEDS Physician Competency Framework as elements of the physician role as a collaborator, manager, and health advocate. Despite a recent shift in the ACGME assessment system to the Next Accreditation System, the framework for assessing milestones within the competencies still requires specific tools to assess SBP.3 Although the conceptual definition of SBP does not easily translate into observable actions or behaviors, the Council of Emergency Medicine Residency Directors (CORD) was able to develop emergency medicine (EM)-specific evaluation domains for SBP. 4 These included specific actions and behaviors that are directly observable and facilitated real-time and summative feedback. However, this set of observable behaviors did overlook some pertinent areas, such as situational awareness and participation in systems improvement. Similarly, by using the Can-MEDS roles of collaborator, manager, and health advocate, Graham et al. also developed a comprehensive list of observable actions that readily translated into assessments. 5This article summarizes the authors' review of the current assessment tools for the SBP competency used in EM and non-EM residencies, both within and outside the United States. As a result of several small group discussions during the breakout session on the assessment of observable learner performance at the 2012 Academic Emergency Medicine (AEM) consensus conference on education research in EM, we developed a research plan for assessment tool development. PROCESS AND CONSENSUSWe reviewed the literature on SBP assessment, searching the Medline and Pubmed databases by combining the terms "systems based" with "evaluat* OR assess*" and "competenc*" as well as combining "simulation" or "portfolio" with "systems based practice," yielding 156 and 309 references, respectively.

show abstract

Section: Ratings and Survey-based Instrumentsmentioning

confidence: 95%

“…Ratings and survey-based instruments: Currently, one global rating scale has validity evidence for the assessment of EM resident performance. 35 Patient surveys and nursing surveys used to provide feedback on resident behavior have not been rigorously tested. Proposed research agenda:…”

Section: Situational Awareness and Teamwork Scoring Systemsmentioning

confidence: 99%

Assessing Systems‐based Practice

Chen¹,

O’Sullivan

Pfennig

et al. 2012

Academic Emergency Medicine

View full text Add to dashboard Cite

show abstract

“…einer Leistung oder eines Verhaltens zu einem spezifischen Ereignis definiert ist, wird als "Evaluation" die standardisierte und formelle Form einer Beurteilung in definierten Zeitabständen verstanden. Auch wenn eine Form der strukturierten Assistenzarztbeurteilung von einigen medizinischen Ausbildungsinstanzen vorgegeben wird, variiert die Implementierung einer solchen Vorgabe im deutschen Sprachraum und anderswo erheblich [3]. Lediglich die Einführung der strukturierten Beobachtungsbögen mit Lernzielformulierungen für eine Patientenuntersuchung ("mini clinical examination", miniCEX) und für selbstständig durchgeführte Eingriffe ("direct observation of procedural skills", DOPS,) durch mehrere "peers" sind erste Maßnahmen einer unreflektierten Ausbildung und einseitigen Beurteilungsinstanzen entgegenzuwirken [4].…”

Section: Feedback Und Evaluationunclassified

Evaluation in der Assistenzarztausbildung

et al. 2015

View full text Add to dashboard Cite

The system was considered to be practical to use and stable in its functionality. The centrally conducted anonymous national survey of residents revealed a noticeable improvement of satisfaction with the institute assessment for the criterion "regular feedback"compared to the national average. Since its implementation the system has been further developed and extended and is now available for other institutions.

show abstract

“…Because the core competencies are each unique in their constructs, they may not be accurately assessed with a single instrument. 22 Reisdorff et al 23 attempted to validate a global rating tool for EM residents. Principal component analysis revealed professionalism to be a complex multidimensional phenomenon to measure, and as such, comprehensive global rating tools may not be ideal tools for assessing professionalism in EM residents.…”

Section: Ratings-and Survey-based Assessment Toolsmentioning

confidence: 99%

Assessing Professionalism: Summary of the Working Group on Assessment of Observable Learner Performance

Rodríguez

Siegelman

Leone

et al. 2012

Academic Emergency Medicine

View full text Add to dashboard Cite

Professionalism is one of the six Accreditation Council on Graduate Medical Education (ACGME) core competencies on which emergency medicine (EM) residents are assessed. However, very few assessment tools exist that have been rigorously evaluated in this population. One goal of the 2012 Academic Emergency Medicine consensus conference on education research in EM was to develop a research agenda for testing and developing tools to assess professionalism in EM residents. A literature review was performed to identify existing assessment tools. Recommendations on future research directions

show abstract

Quantitative Validation of a General Competency Composite Assessment Evaluation

Abstract: By using a structured development process, the authors were able to create valid evaluation items for determining resident acquisition of the ACGME GCs.

Cited by 13 publications

References 5 publications

Assessing Systems‐based Practice

Assessing Systems‐based Practice

Evaluation in der Assistenzarztausbildung

Assessing Professionalism: Summary of the Working Group on Assessment of Observable Learner Performance

Contact Info

Product

Resources

About