Impact of Gender on Clinical Evaluation of Trainees in the Intensive Care Unit

Spring, Jenna; Abrahams, Caroline; Ginsburg, Shiphra; Piquette, Dominique; Guasch, Fernando Martinez; Kiss, Alex; Mehta, Sangeeta

doi:10.34197/ats-scholar.2021-0048oc

Cited by 3 publications

(1 citation statement)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Finally, as with other automated solutions, there is a risk that automated algorithms could paradoxically reinforce ingrained biases rather than mitigate them ( 36 , 37 ). Specifically, care will need to be taken to ensure that this approach does not harm students from disadvantaged backgrounds or those with English as a second language who may use speech patterns that differ from historical norms.…”

Section: Discussionmentioning

confidence: 99%

Initial Development of an Automated Platform for Assessing Trainee Performance on Case Presentations

King¹,

Kahn²,

Brant³

et al. 2022

ATS Scholar

View full text Add to dashboard Cite

Background Oral case presentation is a crucial skill of physicians and a key component of team-based care. However, consistent and objective assessment and feedback on presentations during training are infrequent. Objective To determine the potential value of applying natural language processing, computer software that extracts meaning from text, to transcripts of oral case presentations as a strategy to assess their quality automatically and objectively. Methods We transcribed a collection of simulated oral case presentations. The presentations were from eight critical care fellows and one critical care attending. They were instructed to review the medical charts of 11 real intensive care unit patient cases and to audio record themselves, presenting each case as if they were doing so on morning rounds. We then used natural language processing to convert the transcripts from human-readable text into machine-readable numbers. These numbers represent details of the presentation style and content. The distance between the numeric representation of two different transcripts negatively correlates with the similarity of those two transcripts. We ranked fellows on the basis of how similar their presentations were to the attending’s presentations. Results The 99 presentations included 260 minutes of audio (mean length: 2.6 ± 1.24 min per case). On average, 23.88 ± 2.65 sentences were spoken, and each sentence had 14.10 ± 0.67 words, 3.62 ± 0.15 medical concepts, and 0.75 ± 0.09 medical adjectives. When ranking fellows on the basis of how similar their presentations were to the attending’s presentation, we found a gap between the five fellows with the most similar presentations and the three fellows with the least similar presentations (average group similarity scores of 0.62 ± 0.01 and 0.53 ± 0.01, respectively). Rankings were sensitive to whether presentation style or content information were weighted more heavily when calculating transcript similarity. Conclusion Natural language processing enabled the ranking of case presentations on the basis of how similar they were to a reference presentation. Although additional work is needed to convert these rankings, and underlying similarity scores, into actionable feedback for trainees, these methods may support new tools for improving medical education.

show abstract

Section: Discussionmentioning

confidence: 99%

Initial Development of an Automated Platform for Assessing Trainee Performance on Case Presentations

King¹,

Kahn²,

Brant³

et al. 2022

ATS Scholar

View full text Add to dashboard Cite

show abstract

L’influence du genre des résident·es et des professeur·es sur les évaluations de la formation médicale fondée sur les compétences en anesthésie

Matava

Alam

Kealey

et al. 2023

Can J Anesth/J Can Anesth

View full text Add to dashboard Cite

Purpose Competency-based medical education (CBME) relies on frequent workplace-based assessments of trainees, providing opportunities for conscious and implicit biases to reflect in these assessments. We aimed to examine the influence of resident and faculty gender on performance ratings of residents within a CBME system. Methods This retrospective cohort study took place from August 2017 to January 2021 using resident assessment data from two workplace-based assessments: the Anesthesia Clinical Encounter Assessment (ACEA) and Entrustable Professional Activities (EPAs). Self-reported gender data were also extracted. The primary outcome—gender-based differences in entrustment ratings of residents on the ACEA and EPAs—was evaluated using mixed-effects logistic regression, with differences reported through odds ratios and confidence intervals (α = 0.01). Gender-based differences in the receipt of free-text comments on the ACEA and EPAs were also explored. Results In total, 14,376 ACEA and 4,467 EPA assessments were analyzed. There were no significant differences in entrustment ratings on either assessment tool between men and women residents. Regardless of whether assessments were completed by men or women faculty, entrustment rates between men and women residents were not significantly different for any postgraduate year level. Additionally, men and women residents received strengths-related and actions-related comments on both assessments at comparable frequencies, irrespective of faculty gender. Conclusion We found no gender-based differences in entrustment ratings for both the ACEA and EPAs, which suggests an absence of resident gender bias within this CBME system. Given considerable heterogeneity in rater leniency, future work would be strengthened by using rater leniency-adjusted scores rather than raw scores. Supplementary Information The online version contains supplementary material available at 10.1007/s12630-023-02454-x.

show abstract

Evidence of grade inflation in bachelor of business administration degrees in Turkey for the period from 2002 to 2022

Karadag,

Dortyol

2024

Humanit Soc Sci Commun

View full text Add to dashboard Cite

The study analysed the change in the proportion of business management undergraduate programme graduates in Turkey who received honours (>2.99) over the past two decades. It also investigated the factors that contribute to grade inflation and the distribution of course grades. The research employed various statistical methods, including the random effects estimator, ANOVA, t-test, and ANCOVA, to examine the GPAs and 515,739 student grades from 12,579 courses taken by 46,416 graduates of business management undergraduate programmes at 40 universities between 2002 and 2022. According to our findings, the percentage of undergraduate business management students graduating with honours (>2.99) has increased from 11% in 2002 to 56% in 2022. The most significant increases occurred in 2021 and 2022, which coincided with the COVID-19 pandemic. Our research also revealed grade inflation, with a significant increase of 28% after controlling for variables that may affect graduation grades in Turkish undergraduate business management programmes. The factors that affect course grades include the student’s gender, grade level, language of instruction, instructor title, and course content. According to our findings, it can be concluded that the rapid expansion of undergraduate business management programmes in Turkey has resulted in a decline in qualifications. This, in turn, has led to a rise in grades.

show abstract

Impact of Gender on Clinical Evaluation of Trainees in the Intensive Care Unit

Cited by 3 publications

References 24 publications

Initial Development of an Automated Platform for Assessing Trainee Performance on Case Presentations

Initial Development of an Automated Platform for Assessing Trainee Performance on Case Presentations

L’influence du genre des résident·es et des professeur·es sur les évaluations de la formation médicale fondée sur les compétences en anesthésie

Evidence of grade inflation in bachelor of business administration degrees in Turkey for the period from 2002 to 2022

Contact Info

Product

Resources

About