Validation of a newly developed competency assessment tool for the Posterior Sagittal Anorectoplasty

Abstract

Background: The correction of an anorectal malformation (ARM) is highly complex and relatively infrequent. Simulation training and subsequent assessment of its effectiveness may result in better clinical outcomes. Assessment can be done using a CAT to assess surgical skills of a specific procedure. This study aims to develop and validate a competency assessment tool (CAT) for the posterior sagittal anorectoplasty (PSARP) for the repair of anorectal malformation. Methods: The CAT-PSARP was developed after consultation with experts in the field. The PSARP was divided into five steps, while general skills (tissue and instrument handling) were scored separately. During several pediatric colorectal hands-on courses in 2019 and 2020 participants were asked to voluntarily participate. They performed one PSARP procedure on an ARM simulation model, while being assessed by two objective observers using the CAT-PSARP.Results: A total of 82 participants were enrolled. Intra-class correlation coefficient analysis demonstrated a fair inter-observer agreement for ‘total score of general skills’ (ICC =0.524, p<0.001), a good agreement for ‘total score of specific skills’ (ICC=0.646 , p<0.001) and ‘total score overall’ (ICC=0.669, p<0.001). When comparing the experienced with the inexperienced group a significantly higher score for the experienced group was found for all steps (p=0.013 to p<0.001), except for step 5 ‘anoplasty’ (p=0.440). Conclusion: Good inter-observer agreement was found for the total scores of the CAT-PSARP. It is a suitable objective assessment tool for the overall performance of the included steps of the PSARP for repair of an ARM.