Can We Agree on Expectations and Assessments of Graduating Residents?

Nousiainen, Markuu; Incoll, Ian; Peabody, Terrance D.; Marsh, J. Lawrence

doi:10.2106/jbjs.16.01048

Cited by 18 publications

(22 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The American Board of Orthopaedic Surgery (ABOS) and American Orthopaedic Association's Council of Residency Program Directors (PDs) (American Orthopaedic Association's Council of Residency Directors [AOA/CORD]) continue with their collaboration to develop the knowledge, skills, and behavior project 1 . The behaviors portion of this project deals with the actions that as a whole reflect the resident's degree of “professionalism” 2 .…”

mentioning

confidence: 99%

Use of the Behavior Assessment Tool in 18 Pilot Residency Programs

et al. 2020

Self Cite

View full text Add to dashboard Cite

Update This article was updated on March 17, 2022, because of previous errors on pages 1, 4, 5, 6, and 7. In the Abstract, Results, and Discussion sections, the specificity of the ABOS Behavior Assessment Tool that had read “57%” and “57% (95% CI 52% to 62%)” now reads “51%” and “51% (95% CI 45% to 56%).” In the Abstract, “1,012 evaluators” now reads “1,016 evaluators” and “431 residents” now reads “428 residents.” In the Results section entitled “Evaluation Results per Resident,” the sentence that had read “The domain with the greatest number of residents exhibiting low scores was ethical behavior.” now reads “The domain with the greatest number of residents exhibiting low scores was interaction.” In the Discussion section on pages 6 and 7, the number of residents with low scores in at least one domain that had read “176” now reads “196.” In Table III, the title that had read “Behavior Evaluations Completed by Resident Year in Training Source” now reads “Behavior Evaluations Completed by Resident Year in Training.” In Table VII, the column head that had read “No. of Residents with >2 Low Scores within the Same Domain” now reads “No. of Residents with ≥2 Low Scores within the Same Domain.” Also in Table VII, in columns 2 and 3, the values that had read “26 (6%), 23 (5%), 19 (4%), 18 (4%), and 20 (5%)” now read “32 (8%), 61 (14%), 63 (15%), 57 (13%), and 59 (14%).” In Table VIII, the column head that had read “No. of Baseline Professionalism PD Assessment Low Score Residents (N = 32) Also with Low Scores on the Behavior Tool by at least 2 Evaluators” now reads “No. of Baseline Professionalism PD Assessment Low Score Residents (N = 32) Also with Low Scores on the Behavior Tool.” In Table IX, the title that had read “Number of Low Domain Scores by at least 2 Evaluators for Low Baseline Professionalism PD Assessment Score Residents (n = 32)*” now reads “Number of Low Domain Scores for Low Baseline Professionalism PD Assessment Score Residents (n = 32)*”. Also, the values in the table that had read “6, 3, 2, 2, 6, 13” now read “7, 2, 1, 3, 5, 14.” In Table X, the title that had read “Specificity and Sensitivity of the ABOS Behavior Tool Compared with PD Baseline Assessment for All Participating Residents (n = 440)*” now reads “Specificity and Sensitivity of the ABOS Behavior Tool Compared with PD Baseline Assessment for All Participating Residents (n = 428)*.” Also in Table X, in the right column entitled “PD Baseline Assessment High Score (3 or 4),” the values that had read “176” and “232” now read “196” and “200,” respectively. Finally, a supplementary data file has now been included with the article that contains results that portray resident performance with at least two low scores in one domain by at least two different evaluators. An erratum has been published: JBJS Open Access. 2022 Mar 31;7(1):e20.00103ER. Background: The purpose of this study was to determine the feasibility and evaluate the effectiveness of the American Board of Orthopaedic Surgery Behavior Tool (ABOSBT) for measuring professionalism. Methods: Through collaboration between the American Board of Orthopaedic Surgery and American Orthopaedic Association's Council of Residency Directors, 18 residency programs piloted the use of the ABOSBT. Residents requested assessments from faculty at the end of their clinical rotations, and a 360° request was performed near the end of the academic year. Program Directors (PDs) rated individual resident professionalism (based on historical observation) at the outset of the study, for comparison to the ABOSBT results. Results: Nine thousand eight hundred ninety-two evaluations were completed using the ABOSBT for 449 different residents by 1,016 evaluators. 97.6% of all evaluations were scored level 4 or 5 (high levels of professional behavior) across all of the 5 domains. In total, 2.4% of all evaluations scored level 3 or below reflecting poorer performance. Of 428 residents, the ABOSBT identified 26 of 32 residents who were low performers (2 or more < level 3 scores in a domain) and who also scored “below expectations” by the PD at the start of the pilot project (81% sensitivity and 51% specificity), including 13 of these residents scoring poorly in all 5 domains. Evaluators found the ABOSBT was easy to use (96%) and that it was an effective tool to assess resident professional behavior (81%). Conclusions: The ABOSBT was able to identify 2.4% low score evaluations (<level 3) for all residents. The tool was concordant with the PD for 81% of the residents considered low performers or “outliers” for professional behavior. The 5-domain construct makes it an effective actionable tool that can be used to help develop performance improvement plans for residents. Level of Evidence: Level II

show abstract

mentioning

confidence: 99%

Use of the Behavior Assessment Tool in 18 Pilot Residency Programs

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In the past decade, the academic orthopaedic community slowly began to introduce the concept of competency-based education [25]. The American Board of Orthopaedic Surgery, in cooperation with the Accreditation Council for Graduate Medical Education (ACGME), launched an initiative aimed at defining and assessing the essential clinical, surgical, and professional skills necessary for orthopaedic surgery residents to function as independent orthopaedic surgeons [26]. Despite the wellintentioned proposal, adoption of a competency-based orthopaedic education has been slow and met with resistance, owing primarily to the difficulty of standardizing assessments and the need for "buy-in" from all academic faculty [27].…”

Section: Discussionmentioning

confidence: 99%

Evaluating the Early Impact of the COVID-19 Pandemic on Sports Surgery Fellowship Education

Swiatek¹,

Weiner²,

Alvandi³

et al. 2021

Cureus

View full text Add to dashboard Cite

Purpose The COVID-19 pandemic forced many hospitals to cancel elective surgeries to minimize the risk of viral transmission and ensure the availability of vital health resources. The unintended consequences of this action on the education and training of orthopaedic sports surgeons are unknown. The purpose of this study is to measure the impact of COVID-19 on orthopaedic sports surgery fellows, their education and training, and their readiness for practice. Methods A comprehensive survey was created and distributed to all U.S. fellows and fellowship directors registered with the American Orthopaedic Society for Sports Medicine. Responses were collected between April 22, 2020, and May 5, 2020. Results Fifty-one sports fellows and twenty-nine sports fellowship directors completed the survey. Over 80.4% of fellows reported a greater than 50% decrease in the case volume since the cessation of elective cases. Average hours worked per week decreased by 58.2% during the pandemic. Fellows reported completing an average of 324.6 ± 97.4 cases prior to the COVID-19 crisis and 86.0% expected to complete at least 11% to 25% fewer cases by graduation compared to previous fellows. 87.5% of fellows were not concerned about their ability to complete their fellowship training but more than one-third of fellows voiced concerns to their fellowship directors regarding their readiness for independent practice. Fellowship directors were generally not concerned that COVID-19 would prevent their fellows from completing the fellowship. At least 54.2% are somewhat concerned about the impact of COVID-19 on their future job opportunities. Conclusions The COVID-19 pandemic has universally affected work hours and case volume of sports fellows. Nevertheless, most sports fellows feel prepared to enter practice and are generally supported by the confidence of their fellowship directors. The results of this survey emphasize the importance of the fellowship year in sports training and highlight the future of online education and simulation as useful adjuncts.

show abstract

“… 24 , 25 The American Board of Orthopaedic Surgery, in cooperation with the Accreditation Council for Graduate Medical Education, initiated a project to define and assess the essential knowledge, skills, and behaviors that need to be acquired by orthopedic residents during training in order for them to be competent for independent practice. 24 However, the adoption of competency-based surgical education has been slow and met with resistance due to concerns regarding implementation and lack of normative data to compare across programs. 26 Fellowship training in the era of COVID-19 provides an even more striking example of the importance of a competency-based model and highlights an opportunity to reshape fellow education toward a more substantial and acceptable format than has been achieved through the ACGME’s current offerings.…”

Section: Discussionmentioning

confidence: 99%

Assessing the Early Impact of the COVID-19 Pandemic on Spine Surgery Fellowship Education

Swiatek¹,

Weiner²,

Butler³

et al. 2020

Clinical Spine Surgery: A Spine Publication

View full text Add to dashboard Cite

Study Design: This was a cross-sectional study. Objective: The objective of this study is to report the impact of COVID-19 on spine surgery fellow education and readiness for practice. Summary of Background Data: COVID-19 has emerged as one of the most devastating global health crises of our time. To minimize transmission risk and to ensure availability of health resources, many hospitals have cancelled elective surgeries. There may be unintended consequences of this decision on the education and preparedness of current surgical trainees. Materials and Methods: A multidimensional survey was created and distributed to all current AO Spine fellows and fellowship directors across the United States and Canada. Results: Forty-five spine surgery fellows and 25 fellowship directors completed the survey. 62.2% of fellows reported >50% decrease in overall case volume since cancellation of elective surgeries. Mean hours worked per week decreased by 56.2%. Fellows reported completing a mean of 188.4±64.8 cases before the COVID-19 crisis and 84.1% expect at least an 11%–25% reduction in case volume compared with previous spine fellows. In all, 95.5% of fellows did not expect COVID-19 to impact their ability to complete fellowship. Only 2 directors were concerned about their fellows successfully completing fellowship; however, 32% of directors reported hearing concerns regarding preparedness from their fellows and 25% of fellows were concerned about job opportunities. Conclusions: COVID-19 has universally impacted work hours and case volume for spine surgery fellows set to complete fellowship in the middle of 2020. Nevertheless, spine surgery fellows generally feel ready to enter practice and are supported by the confidence of their fellowship directors. The survey highlights a number of opportunities for improvement and innovation in the future training of spine surgeons. Level of Evidence: Level III.

show abstract

Can We Agree on Expectations and Assessments of Graduating Residents?

Cited by 18 publications

References 10 publications

Use of the Behavior Assessment Tool in 18 Pilot Residency Programs

Use of the Behavior Assessment Tool in 18 Pilot Residency Programs

Evaluating the Early Impact of the COVID-19 Pandemic on Sports Surgery Fellowship Education

Assessing the Early Impact of the COVID-19 Pandemic on Spine Surgery Fellowship Education

Contact Info

Product

Resources

About