Use of the “Step‐back” Method for Education Research Consultation at the National Level: A Pilot Study

Abstract

Background: There are a limited number of emergency medicine (EM) physicians with expertise in education research. The Harvard Macy "step-back" method is an emerging model utilized to gather group feedback. Despite its use in multiple educational settings, there are little published data demonstrating effectiveness.Objectives: Our objective was to create and evaluate a national faculty development session providing consultation in education research utilizing the step-back method.Methods: This was a pilot study. EM experts in education research from across the country served as facilitators for a faculty development session held at the 2018 Council of Emergency Medicine Residency Directors Academic Assembly. Small groups consisting of two or three facilitators and one or two participants were formed and each participant underwent a step-back consultation for their education research study. Participants wrote their study question before and after the session. After the session, facilitators and participants completed an evaluative survey consisting of multiple-choice, Likert-type, and free-response items. Descriptive statistics were reported. Qualitative analysis using a thematic approach was performed on free-response data. Participant study questions were assessed by the PICO (population, intervention, comparison, outcome) and FINER (feasible, interesting, novel, ethical, relevant) criteria. Both scales were evaluated using a two-way randomconsistency intraclass correlation. Before and after scores were evaluated with a paired t-test.Results: Twenty-four facilitators and 13 participants completed the step-back session. Evaluations from 20 facilitators and nine participants were submitted and analyzed. Sixteen of 20 facilitators felt that the step-back method "greatly facilitated" their ability to share their education research expertise. All facilitators and participants recommended that the session be provided at a future academic assembly. Regarding suggestions for improvement, qualitative analysis revealed three major themes: praise for the session, desire for additional time, and a room set up more conducive to small group work. Seven of nine responding participants felt that the session was "very valuable" for improving the strength of their study methods. Qualitative analysis regarding change in study as a result of the step-back session yielded four major themes: refinement of study question, more specific outcomes and measurements, improvement in study design, and greater understanding of study limitations. Both FINER and PICO scale comparisons showed improvement pre-and postintervention (PICO 60% relative increase; FINER 16% relative increase). Neither achieved statistical significance (PICO t(5) = -1.835, p = 0.126; and FINER t(5) = -1.305, p = 0.249).

Educational approaches for social accountability in health professions training: a scoping review protocol

“…To complete the scoping study, we will use the Harvard Macy "step-back" method to gather knowledge user group feedback. 20 We plan to conduct three focus groups at three HPE faculties in Canada, each comprising six to eight participants purposively recruited 21 from various partner groups (e.g., learners, community members, faculty, and leaders in social accountability). Our objective is to present our findings and intentionally withdraw (i.e., "step-back") from the discussion, enabling the group members to engage in open deliberation and discuss implications.…”

The Impact of Women’s Agency on Accessing and Using Maternal Healthcare Services: A Systematic Review and Meta-Analysis

“…These criteria investigated if the research question is Feasible (F), Interesting (I), Novel (N), Ethical (E), and Relevant (R) (Table 1). To formulate the research question, the authors investigated these criteria and rated each question as "yes" or "no" [38]. Regarding the research question in this study, the response for all components was "yes".…”

Virtual Mentoring: Two Adaptive Models for Supporting Early‐career Simulation Investigators in the Era of Social Distancing

“…The CS intervenes earlier at steps 2 and 3, before data collection occurs, and focuses on research question refinement and methods analysis. We used the Harvard Macy “step‐back” method for educational research 10 as a model for running the e‐consult sessions and adapted it into a virtual format to gather group feedback from the consultants in a structured fashion. This method allowed presenters to step back after briefly presenting their project proposal and listen to experts who then analyzed and refined the project through active dialog.…”