Resident and attending perceptions of direct observation in internal medicine: a qualitative study

Abstract

OBJECTIVES Direct observation is the foundation of assessment and learning in competency-based medical education (CBME). Despite its importance, there is significant uncertainty about how to effectively implement frequent and high-quality direct observation. This is particularly true in specialties where observation of nonprocedural skills is highly valued and presents unique challenges. It is therefore important to understand perceptions of direct observation to ensure successful acceptance and implementation. In this study, we explored perceptions of direct observation in internal medicine.METHODS We interviewed internal medicine attending physicians (n = 9) and residents (n = 8) at the University of Toronto, purposively sampled for diversity. Using a constructivist grounded theory approach, constant comparative analysis was performed to develop a framework to understand perceptions of direct observation on the clinical teaching units. RESULTS Participants articulated a narrow perception of what constitutes directobservation, in contrast to their own descriptions of skills that were observed. This resulted in the perception that certain valuable skills that participants felt were routinely observed were nonetheless not 'directly observable', such as clinical reasoning, observed through case presentations and patient care discussions. Differentiating direct observation from informal observation led to overestimation of the time and resource requirements needed to enhance direct observation, which contributed to scepticism and lack of engagement related to CBME implementation.CONCLUSIONS In an internal medicine training programme, perceptions of what constitutes direct observation can lead to under-recognition and hinder acceptance in workplace-based assessment and learning. Our results suggest a reframing of 'direct observation' for residents and attending physicians, by explicitly identifying desired skills in non-procedurally-based specialties. These findings may help CBME-based training programmes improve the process of direct observation, leading to enhanced assessment and learning.

Perceptions of scheduled vs. unscheduled directly observed visits in an internal medicine residency outpatient clinic

“…Faculty educators are increasingly encouraged to use more direct observation of authentic clinical work to assess the skills of learners [1]. A growing body of literature suggests direct observation as the foundation and ideal standard for formative assessment, but to what extent a scheduled observation and/or in-room observer modifies those assessments is unknown [1][2][3][4][5]. The Hawthorne effect describes a change in behavior in response to observation and assessment [6].…”

Aligning direct observation and assessment

“…They are embedded in the daily workflow of all disciplines. Despite their prevalence, however, oral case presentations are underutilised for assessment because it is unclear how they can be used effectively as assessment tools . A critical step forward in this regard is to understand which skills should and should not be assessed using oral case presentations.…”

“…Direct observation can include any situation in which a supervisor watches a trainee complete a clinical task rather than simply a situation in which a trainee is watched while interacting with a patient. Broadening the definition in this way leads us to consider oral case presentations as opportunities to ‘directly observe’ clinical reasoning in ways that may actually provide greater insight into the trainee's ability in this particular skill than can be achieved when a narrow definition of direct observation in which the supervisor is expected to observe the trainee with a patient is treated as sacrosanct . In essence, it is not that ‘direct observation’ of a trainee–patient encounter necessarily offers a better assessment of the trainee's competence relative to the oral case presentation.…”

“…Other activities that can be viewed as either proxies or opportunities for direct observation according to the skill that needs to be assessed include debriefing following a resuscitation, relaying information provided by a consultant and documenting procedure notes. None of these activities involve direct observation of a trainee–patient encounter, but they can inform entrustment decisions through more direct observation of valued competencies . Supervisors must understand this distinction in order to deliberately choose what to assess when observing any given activity.…”

Direct observation in postgraduate training: making it happen and making it work

“…In this issue, Gauthier et al . investigate residents’ and attending physicians’ perceptions of direct observation before transitioning to a competency‐based approach in their postgraduate internal medicine training programme.…”