How are we assessing competencies? Experience report of an intervention project in the evaluation instruments of an intensive care medicine residency program

Abstract

Introduction: Competency-based teaching in medical residencies has evidenced the mismatch between traditional assessment processes and the educational objectives of pedagogical projects aligned with the competency matrices of each specialty. The competency matrix for the Intensive Care Medicine Residency Program (3-year direct admission) was approved in 2021. The objective of this article is to describe an experience report of an intervention project in the performance assessment instruments of residents attending the Intensive Care Medicine Residency Program at a university hospital in São Luís-Maranhão. Experience Report: After organizing the study and working group for the intervention, the object “skills assessment tools” was chosen and the Intensive Care Medicine Residency Program was selected. Initially, a questionnaire was applied to all preceptors and residents working in the intensive care unit (ICU) setting with the aim of evaluating their perceptions regarding the current assessment tool, following the guiding question: does the current assessment meet the conception of the program translated by the competency matrix of the National Medical Residency Commission? Discussion: Although the majority of the preceptors and residents considered that the evaluation methods met the Program design, there were weaknesses in relation to feedback and evaluation of the residents’ performance. As an intervention, we proposed adapting the existing tool, making it adequate to the performance predicted in the specialty competency matrix with formalization of feedback and introduction of performance assessment in a real scenario using the Mini Clinical Assessment Exercise (Mini-CEX). Conclusion: The boundaries between assessment and learning are blurred. Based on indicators on the perception of preceptors and residents of weaknesses in the assessment used for a long time, an intervention was proposed to modify the assessment instruments with the aim of adapting/improving the assessment of skills.