BACKGROUND: Attending evaluations are commonly used to evaluate residents. OBJECTIVES: Evaluate the quality of written feedback of internal medicine residents. DESIGN: Retrospective. PARTICIPANTS: Internal medicine residents and faculty at the Medical College of Wisconsin from 2004 to 2012. MAIN MEASURES: From monthly evaluations of residents by attendings, a randomly selected sample of 500 written comments by attendings were qualitatively coded and rated as high-, moderate-, or low-quality feedback by two independent coders with good inter-rater reliability (kappa: 0.94). Small group exercises with residents and attendings also coded the utterances as high, moderate, or low quality and developed criteria for this categorization. In-service examination scores were correlated with written feedback. KEY RESULTS: There were 228 internal medicine residents who had 6,603 evaluations by 334 attendings. Among 500 randomly selected written comments, there were 2,056 unique utterances: 29 % were coded as nonspecific statements, 20 % were comments about resident personality, 16 % about patient care, 14 % interpersonal communication, 7 % medical knowledge, 6 % professionalism, and 4 % each on practice-based learning and systems-based practice. Based on criteria developed by group exercises, the majority of written comments were rated as moderate quality (65 %); 22 % were rated as high quality and 13 % as low quality. Attendings who provided high-quality feedback rated residents significantly lower in all six of the Accreditation Council for Graduate Medical Education (ACGME) competencies (p <0.0005 for all), and had a greater range of scores. Negative comments on medical knowledge were associated with lower in-service examination scores. CONCLUSIONS: Most attending written evaluation was of moderate or low quality. Attendings who provided highquality feedback appeared to be more discriminating, providing significantly lower ratings of residents in all six ACGME core competencies, and across a greater range. Attendings' negative written comments on medical knowledge correlated with lower in-service training scores.

USMLE Step 1 and ITE scores have a modest correlation with board scores. Failing Step 1 or scoring in the bottom quartile of the ITE increased the risk of failing the boards. What effective intervention, if any, program directors may use with at-risk residents is a question deserving further research.