BackgroundTo reduce the incidence of hypoxic brain injuries among newborns a national cardiotocography (CTG) education program was implemented in Denmark. A multiple-choice question test was integrated as part of the program. The aim of this article was to describe and discuss the test development process and to introduce a feasible method for written test development in general.MethodsThe test development was based on the unitary approach to validity. The process involved national consensus on learning objectives, standardized item writing, pilot testing, sensitivity analyses, standard setting and evaluation of psychometric properties using Item Response Theory models. Test responses and feedback from midwives, specialists and residents in obstetrics and gynecology, and medical and midwifery students were used in the process (proofreaders n = 6, pilot test participants n = 118, CTG course participants n = 1679).ResultsThe final test included 30 items and the passing score was established at 25 correct answers. All items fitted a loglinear Rasch model and the test was able to discriminate levels of competence. Seven items revealed differential item functioning in relation to profession and geographical regions, which means the test is not suitable for measuring differences between midwives and physicians or differences across regions. In the setting of pilot testing Cronbach’s alpha equaled 0.79, whereas Cronbach’s alpha equaled 0.63 in the setting of the CTG education program. This indicates a need for more items and items with a higher degree of difficulty in the test, and illuminates the importance of context when discussing validity.ConclusionsTest development is a complex and time-consuming process. The unitary approach to validity was a useful and applicable tool for development of a CTG written assessment. The process and findings supported our proposed interpretation of the assessment as measuring CTG knowledge and interpretive skills. However, for the test to function as a high-stake assessment a higher reliability is required.Electronic supplementary materialThe online version of this article (doi:10.1186/s12909-017-0915-2) contains supplementary material, which is available to authorized users.

Introduction Studies indicate an association between errors in cardiotocography (CTG) management and hypoxic brain injuries among newborns. Continuing professional education is recommended. We aimed to examine whether the implementation of a national interprofessional CTG education program in Denmark was associated with a decrease in risk of fetal hypoxia measured by umbilical cord pH < 7.00, 5‐minute Apgar score <7 or neonatal therapeutic hypothermia. As a secondary aim, we assessed whether the educational intervention was associated with an increase in operative deliveries. Material and methods We conducted a historical cohort study from 2009 to 2015 including all intended vaginal deliveries with liveborn singletons in cephalic presentation and gestational age ≥37 weeks. Data were retrieved from the Medical Birth Register and the National Patient Register. The study period was divided in three: pre‐implementation (2009‐2012), implementation (2013) and post‐implementation (2014‐2015). Using logistic regression we estimated odds ratios (OR) of fetal hypoxia outcomes using the pre‐implementation period as reference. Analyses were adjusted for potential maternal, neonatal and delivery‐associated confounders. Missing data were accounted for by multiple imputation. Results In all, 331 282 deliveries were included. Overall risks of pH < 7.00, Apgar score <7 and therapeutic hypothermia were respectively 0.45%, 0.58% and 0.06%. Adjusted OR in the post‐implementation period were 1.12 (95% confidence interval [CI] 1.00‐1.26), 0.99 (95% CI 0.90‐1.10) and 1.34 (95% CI 0.99‐1.82) for the three outcomes, respectively. The pH missingness equaled 12.4%. Odds of emergency cesarean section was unaltered, whereas the odds of assisted vaginal delivery decreased by 14% (0.86, 95% CI 0.84‐0.89). Conclusions Healthcare professionals are considered the weakest link of CTG technology. We did not find that increasing healthcare professionals’ CTG interpretation skills affected the risk of fetal hypoxia. Missing data for pH values were substantial and represent a limitation of the study. We cannot with certainty rule out that missingness masked a true effect of the intervention. Our study indicates that assisted vaginal deliveries can be decreased without an increased risk of fetal hypoxia. Dilution of effect in a complex clinical setting, rare outcomes, insufficient intervention and a possible overestimation of the impact of errors in CTG management might explain the lack of effect.

Content and construct validity and reliability were acceptable. The presented template for the development of this MCQ test could be useful to others when developing knowledge tests and may enhance the overall quality of test development.