Objectives. To develop and establish validity for a grading rubric to evaluate diabetes SOAP note writing on primary care (PC) advanced pharmacy practice experiences (APPEs), as well as to assess reliability and student perceptions of the rubric.Methods. Ten PC APPE faculty members collaborated to develop a rubric to provide formative and summative feedback on three written SOAP notes per APPE student over a 10-month period. Correlation analyses were conducted between rubric scores and three criterion variables to assess criterion-related validity: APPE grades, Pharmaceutical Care Ability Profile Scores, and Global Impression Scores. Interrater and intra-rater reliability testing were completed using Cohen's kappa and Intraclass Correlation Coefficients (ICC). Student perceptions were assessed through an anonymous student survey.Results. A total of 51 students and 167 SOAP notes were evaluated using the final rurbric. The mean score significantly increased from the first to second SOAP note (p<.001) and from the first to third SOAP note (p<.001). Statistically significant positive correlations were found between final rubric scores and criterion variables. The ICC for inter-rater reliability was fair (.59) for final rubric scores and excellent for intra-rater reliability (.98-1.00). A majority of students responded that the rubric improved their ability and confidence to write SOAP notes.
Conclusions.The findings provide evidence that the rubric developed may be used to make valid decisions about students' SOAP note writing ability and may increase their confidence in this area. The use of the rubric allows for greater reliability among multiple graders, supporting grading consistency.