Inaccuracy of the Global Assessment Score in the Emergency Medicine Standard Letter of Recommendation

Oyama, Leslie C.; Kwon, Manon; Fernandez, Jorge; Fernández-Frackelton, Madonna; Campagne, Danielle; Castillo, Edward M.; Lin, Michelle

doi:10.1111/j.1553-2712.2010.00882.x

Cited by 21 publications

(22 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The use of SLORs may represent a more efficient and dependable replacement for this less reliable tool for candidate assessment in pediatric otolaryngology fellowship selection. The SLOR created by the Council of Emergency Medicine Residency Directors has proven to be an effective means of curtailing the variety of information presented in an NLOR for residency applicants,8, 9 despite the fact that a recent report demonstrated that the global assessment score is only accurate 26% of the time 24. The higher mean and median rankings of the SLOR for writer background, applicant comparison to contemporaries, and applicant qualifications for pediatric otolaryngology in the SLOR may be indicative of the fact that some of the NLORs did not include this information.…”

Section: Discussionmentioning

confidence: 99%

Standardized letter of recommendation for pediatric fellowship selection

et al. 2011

View full text Add to dashboard Cite

Objectives/Hypothesis: To develop a pediatric otolaryngology fellowship selection standardized letter of recommendation (SLOR). Study Design: SLOR and narrative letter of recommendation (NLOR) comparison study. Methods: An SLOR was created to investigate qualities desired in fellows using five content‐based categories: writer background, comparison of the applicant to contemporaries and predecessors, applicant's qualifications for pediatric otolaryngology, a global assessment of the applicant, and a summary statement about the applicant. In phase I, the SLORs were completed, including writing time, by the applicant's pediatric otolaryngology chief. In phase II, letters were ranked on Likert‐type scales for the content‐based categories, reviewer's overall ranking, and ease of review by six otolaryngologists. Reviewers recorded time needed to review each letter. Results: Nineteen SLORs (73%) were collected. Mean writing time was 8.84 ± 3.87 minutes. Interrater reliability was higher on the SLORs in the content‐based sections and the overall ranking. Ranking times were lower on the SLORs. Mean and median rankings were higher on the SLORs for writer background, comparison of the applicant to contemporaries and predecessors, applicant's qualifications for pediatric otolaryngology, and ease of review; mean global assessment of the applicant, summary statement about the applicant, and overall rankings were lower on the SLORs. Conclusions: To our knowledge, this is the first inquiry using an SLOR developed for otolaryngology. SLORs are an alternative to NLORs for fellowship selection that offers improved reliability and efficiency. Further investigation using SLORs in otolaryngology residency selection is merited.

show abstract

Section: Discussionmentioning

confidence: 99%

Standardized letter of recommendation for pediatric fellowship selection

et al. 2011

View full text Add to dashboard Cite

show abstract

“…Applicants are evaluated by residency programs based on various components of their application including United States Medical Licensing Examination (USMLE) scores, the dean’s performance evaluation, clinical rotation grades, extracurricular experiences, the medical school’s reputation, and letters of recommendation 1–4…”

Section: Introductionmentioning

confidence: 99%

Analysis of the Evaluative Components on the Standard Letter of Recommendation (SLOR) in Emergency Medicine

Grall¹,

Hiller²,

Stoneking³

2014

WestJEM

View full text Add to dashboard Cite

IntroductionThe standard letter of recommendation in emergency medicine (SLOR) was developed to standardize the evaluation of applicants, improve inter-rater reliability, and discourage grade inflation. The primary objective of this study was to describe the distribution of categorical variables on the SLOR in order to characterize scoring tendencies of writers.MethodsWe performed a retrospective review of all SLORs written on behalf of applicants to the three Emergency Medicine residency programs in the University of Arizona Health Network (i.e. the University Campus program, the South Campus program and the Emergency Medicine/Pediatrics combined program) in 2012. All “Qualifications for Emergency Medicine” and “Global Assessment” variables were analyzed.Results1457 SLORs were reviewed, representing 26.7% of the total number of Electronic Residency Application Service applicants for the academic year. Letter writers were most likely to use the highest/most desirable category on “Qualifications for EM” variables (50.7%) and to use the second highest category on “Global Assessments” (43.8%). For 4-point scale variables, 91% of all responses were in one of the top two ratings. For 3-point scale variables, 94.6% were in one of the top two ratings. Overall, the lowest/least desirable ratings were used less than 2% of the time.ConclusionsSLOR letter writers do not use the full spectrum of categories for each variable proportionately. Despite the attempt to discourage grade inflation, nearly all variable responses on the SLOR are in the top two categories. Writers use the lowest categories less than 2% of the time. Program Directors should consider tendencies of SLOR writers when reviewing SLORs of potential applicants to their programs.

show abstract

“…7 While there are many benefits of the SLOE, 1,3,9,10 it is unclear whether the SLOE can predict future performance. [11][12][13] The SLOE ranks students in various aspects of their EM clerkship performance (e.g., work ethic, team work, anticipated level of guidance needed). 9 The global assessment section (section C) of the SLOE places the student into a quantile for two questions.…”

mentioning

confidence: 99%

Does Applicant Gender Have an Effect on Standardized Letters of Evaluation Obtained During Medical Student Emergency Medicine Rotations?

Andrusaitis

Clark

Saadat

et al. 2019

AEM Education and Training

View full text Add to dashboard Cite

Background: The standardized letter of evaluation (SLOE) in emergency medicine (EM) is one of the most important items in a student's application to EM residency and replaces narrative letters of recommendation. The SLOE ranks students into quantile categories in comparison to their peers for overall performance during an EM clerkship and for their expected rank list position. Gender differences exist in several assessment methods in undergraduate and graduate medical education. No authors have recently studied whether there are differences in the global assessment of men and women on the SLOE.Objectives: The objective of this study was to determine if there is an effect of student gender on the outcome of a SLOE. Methods:This was a retrospective observational study examining SLOEs from applications to a large urban, academic EM residency program from 2015 to 2016. Composite scores (CSs), comparative rank scores (CRSs), and rank list position scores (RLPSs) on the SLOE were compared for female and male applicants using Mann-Whitney U-test.Results: From a total 1,408 applications, 1,038 applicants met inclusion criteria (74%). We analyzed 2,092 SLOEs from these applications. Female applicants were found to have slightly lower and thus better CRSs, RLPSs, and CSs than men. The mean CRS for women was 2.27 and 2.45 for men (p < 0.001); RLPS for women was 2.32 and 2.52 for men (p < 0.001) and CS was 4.59 for women and 4.97 for men (p < 0.001).Conclusions: Female applicants have somewhat better performance on the EM SLOE than their male counterparts.From the

show abstract

Inaccuracy of the Global Assessment Score in the Emergency Medicine Standard Letter of Recommendation

Cited by 21 publications

References 6 publications

Standardized letter of recommendation for pediatric fellowship selection

Standardized letter of recommendation for pediatric fellowship selection

Analysis of the Evaluative Components on the Standard Letter of Recommendation (SLOR) in Emergency Medicine

Does Applicant Gender Have an Effect on Standardized Letters of Evaluation Obtained During Medical Student Emergency Medicine Rotations?

Contact Info

Product

Resources

About