Development and Initial Validation of a Project-Based Rubric to Assess the Systems-Based Practice Competency of Residents in the Clinical Chemistry Rotation of a Pathology Residency

Abstract

Context.-Systems-based practice (SBP) is 1 of 6 core competencies required in all resident training programs accredited by the Accreditation Council for Graduate Medical Education. Reliable methods of assessing resident competency in SBP have not been described in the medical literature.Objective.-To develop and validate an analytic grading rubric to assess pathology residents' analyses of SBP problems in clinical chemistry.Design.-Residents were assigned an SBP project based upon unmet clinical needs in the clinical chemistry laboratories. Using an iterative method, we created an analytic grading rubric based on critical thinking principles. Four faculty raters used the SBP project evaluation rubric to independently grade 11 residents' projects during their clinical chemistry rotations. Interrater reliability and Cronbach a were calculated to determine the reliability and validity of the rubric. Project mean scores and range were also assessed to determine whether the rubric differentiated resident critical thinking skills related to the SBP projects.Results.-Overall project scores ranged from 6.56 to 16.50 out of a possible 20 points. Cronbach a ranged from 0.91 to 0.96, indicating that the 4 rubric categories were internally consistent without significant overlap. Intraclass correlation coefficients ranged from 0.63 to 0.81, indicating moderate to strong interrater reliability.Conclusions.-We report development and statistical analysis of a novel SBP project evaluation rubric. The results indicate the rubric can be used to reliably assess pathology residents' critical thinking skills in SBP.

Methodological Rigor in Laboratory Education Research

“…In their attempt at establishing rigor, they acknowledge that surveys could cause response fatigue, so redundancy should be avoided. Likewise, Vitek and colleagues [33] also used an iterative method in their design and development of a project-based rubric to assess clinical chemistry residents. To compare with the qualitative research in their phenomenological study on student experiences of reform in a general chemistry laboratory, Chopra and colleagues [34] emphasized the inductive and iterative processes involving five researchers to reduce verbal data into the essence and meaning of experiences.…”

Initial Validation of a Technical Writing Rubric for Engineering Design

“…18 and 19 and a study performed on initial validation of a project based rubric in Ref. 20 [18,19,20].…”