The Residency Performance Index: An Effort at Residency Quality Assessment and Improvement in Family Medicine

Abstract

Background Residency programs are increasingly being asked to defend their quality, and that of the residents they produce. Yet “residency quality” is a construct that has not been well defined, with no accepted standards other than meeting accreditation standards. In 2009, the Association of Family Medicine Residency Directors developed a strategic plan that included the goal of raising the quality of family medicine training. Objective We describe the development of this quality improvement tool, which we called the residency performance index (RPI), and its first year of use by family medicine residency programs. We describe the use of the tool as a “dashboard” to facilitate program self-improvement. Intervention Using program metrics specific to family medicine training, and benchmark criteria for these metrics, the RPI was launched in 2012 to help programs identify strengths and areas for improvement in their educational activities and resident clinical experiences that could be tracked and reviewed as part of the annual program evaluation. Results Approximately 100 program directors began using the tool and 70 finished the process, and were provided aggregate data. Initial review of this experience revealed difficulties with collecting data, and lack of information on graduates' scope of practice. It also showed the potential usefulness of the tool as a program improvement mechanism. Conclusions The RPI is a new quality improvement tool for family medicine residency programs. Although some initial challenges need to be addressed, it has the promise to aid family medicine residency in its internal improvement efforts.

A large survey among European trainees in clinical microbiology and infectious disease on training systems and training adequacy: identifying the gaps and suggesting improvements

“…In fact, not all residency programs are accredited. Accreditation of a training program is needed in order to improve quality and facilitate exchange of trainees between institutions [5, 19]. …”

Six Years’ Experience Using Specialty National Residency Index: An Early Dashboard to Document Change Over Time

“…A convention of red, yellow, and green was chosen for the RPI, with green representing achievement of excellence in quality targets; yellow denoting adequate program quality for accreditation purposes but with room for improvement or caution; and red indicating metrics below accreditation standards, national norms, or targets promoted by family medicine professional organizations, based on published requirements or literature (Table 1). 13,14 Analysis of the aggregate data for the 6 years (2012-2017) that the survey was active revealed that 646 program-years of data from 248 unique FM residency programs could be reliably extracted. Adjusting for minor variations in the wording of survey questions over the years, the data can be grouped into categories of questions: (1) program accreditation and program leadership status, (2) family medicine center (FMC) resident clinical experience, (3) hospitalbased resident clinical experience, (4) procedural training, (5) residency program scholarship, and (6) graduate scope of practice.…”

“…11,12 We have previously described the process to develop this tool and its initial successes and challenges. 13,14 In 2018, the AFMRD Board made the decision to sunset the RPI due to insufficient return on investment as demonstrated by increasing software costs and staff time coupled with lack of growth in participation and only modest improvement in quality. This paper describes the lessons learned from an analysis of 6 years of data collected while the RPI was in use that can inform future quality and accreditation efforts implemented on a national level.…”

What if Child Maltreatment Were Childhood Leukemia?: Lessons from the Halls of the Hospital

“…Finally, metrics for evaluation of success in training workers in Child Protective Services are lacking in that they typically address the training itself rather than competency and confidence. The residency program index (RPI) was implemented in 2012 to help programs identify strengths and areas for improvement in their educational activities and resident clinical experiences that could be tracked and reviewed as part of the annual program evaluation (Hoekzema et al 2014). This type of index could be developed for state child welfare agencies and maintained on a federal level for accountability.…”