Measuring change in knowledge acquisition of Rwandan residents: using the American Board of Pediatrics International In-Training Examination (I-ITE) as an independent tool to monitor individual and departmental improvements during the Human Resources for Health program: an observational study

Abstract

Background Rwanda is the only African country to use the pediatric International In-Training Examination (I-ITE). The objectives of this study were to use the scores from the I-ITE to outline the baseline level of knowledge of Rwandan residents entering the pediatric residency and the trends in knowledge acquisition from 2012 to 2018, during the Human Resources for Health (HRH) Program, an education partnership between the Rwanda Ministry of Health and a consortium of US universities. Methods A retrospective descriptive analysis of the I-ITE exam scores, taken by all Rwandan pediatric residents for five of the six academic years of the study period. Individual resident scores were weighted using the non-Rwandan I-ITE sites to minimise confounding from annual variations in exam difficulty. Statistical analysis included descriptives with ANOVA to compare variation in annual mean scores. Results Eighty-four residents took 213 I-ITE exam sittings over the five exam cycles. The mean weighted I-ITE score of all residents increased from 34% in 2013 to 49% ( p < 0.001) in 2018. The 32-point gap between the mean US-ITE and Rwandan I-ITE score in 2012–2013 was reduced to a 16-point gap in 2017–2018. First year resident (PG1) scores, which likely reflect the knowledge level of undergraduate medical students entering the residency program, increased from 34.8 to 44.3% ( p = 0.002) between 2013 and 2018. Conclusions The I-ITE is an independent, robust tool, measuring both learners and the institutional factors supporting residents. This is the first study to demonstrate that the I-ITE can be used to monitor resident knowledge acquisition in resource-limited settings, where assessment of resident knowledge can be a major challenge facing the academic medicine community. The significant increase in I-ITE scores between 2012 and 18 reflects the substantial curricular reorganisation accomplished through collaboration between Rwandan and US embedded faculty and supports the theory that programs such as HRH are highly effective at improving the quality of residency programs and undergraduate medical education.

Experiences of the International In-Training Examination (I-ITE) by Rwandan pediatric residents – a mixed-methods description of candidate feedback

“…We sought to gain a rich and detailed understanding of the experiences, attitudes and perceptions of the residents, and for the results to emerge from the data. Our previous paper has demonstrated that the I-ITE exam is a valuable tool to measure knowledge acquisition in Rwandan residents 5 , whereas this current paper, using qualitative methods, gave the opportunity to understand the residents' experience and the cultural relevance to them as end-users.…”

“…In Rwanda, there is a single pediatric residency program graduating approximately five to 18 pediatricians per year. Rwanda is the first sub-Saharan nation to employ the pediatric I-ITE to provide formative feedback to individual residents and to give feedback to the faculty on the overall performance of residents within the residency program 5 . The I-ITE, however, is a formative assessment based on the American ITE examination, and though adapted it is being utilised in a very different cultural and medical context in terms of both the populations, patients and pathologies encountered, but also in respect to the language skills, learning styles and asseessment experience of the residents undertaking the assessment.…”

“…The I-ITE was implemented into the UR pediatric residency program from 2012 to 2018 by the faculty of UR and the Human Resources for Health (HRH) program 5,10,11 . A full description of the implementation of the I-ITE in Rwanda, and the performance of the residents in this assessment, has been previously described in the literature 5,12 .…”

“…The I-ITE is annually reviewed by physicians (a reviewer and 1-2 medical editors), in an attempt to ensure current/relevant information, and, where possible, to remove any "Americanisms", terminology, etc. The I-ITE was piloted with one training program in Lebanon in 2008, and made available to all countries in 2009 5 . The I-ITE has been shown to be a tool that can measure resident knowledge acquisition and institutional factors supporting residents in a resource-limited country, but has not be studied otherwise.…”

Peer-teaching at the University of Rwanda - a qualitative study based on self-determination theory

“…The PI had not previously undertaken qualitative research and was therefore supported by a faculty member (PC) experienced in qualitative research. PC supervised and co-authored the work and is a male medical doctor trained in the United Kingdom, and at the time of the study was an associate professor in pediatrics at Yale University, USA, working on the Human Resources to Health Programme in Rwanda [ 37 , 38 ]. PC has received training in qualitative methods during his Masters in Child Health and has contributed to previous qualitative research in Rwanda (Yale) and the United Kingdom (UoL).…”