The efficacy and usefulness of software carpentry training: a follow-up cohort study

Abstract

Software Carpentry was founded in 1998 as a volunteer initiative, with the goal of improving scientific productivity and research reliability by teaching researchers the foundational coding skills required to perform reproducible science. The initiative was sparked by the increasing importance of programming skills in scientific research, as data volumes and availability grew, and research projects began to increase in scale and complexity. Two-day Software Carpentry workshops, taught by volunteer instructors, introduce researchers to software development practices such as task automation, programming, and version control. This thesis investigates whether Software Carpentry has been effective in achieving its goal of introducing researchers to the software development practices that underpin reproducible science.Despite the shift to data-intensive, computational science over the past decade, most Australian postgraduate students and early career researchers are not offered any basic introduction to, nor training in, programming or data analysis skills. Many researchers lack the computational skills needed to automate repetitive tasks, or to count, mine, analyse, manage, visualise, and clean up data. Many have little awareness of how computation might be employed to speed up data analysis or data processing.Software Carpentry was developed to address this skills gap, and to foster a culture of reproducible science. This work investigates whether Software Carpentry has added genuine value to researchers' methodologies and practices. It does so through a two-stage process: a Web-based survey assessing attendees' satisfaction with workshops, and follow-up interviews. By combining quantitative and qualitative methods, evidence has been assembled of the effects workshop attendance has had on attendees' adoption of coding and other computational skills, and on the changes many have subsequently made to their research practice.Survey feedback indicated that satisfaction with workshops was high, with a median score of 6 on a 1-7 scale. Many respondents had not only applied the skills learned, but had made multiple changes to their research practices. Interviews confirmed the survey findings, and revealed that the way the workshops were taught had had as much, if not more, impact as what was taught -the positive learning environment of Software Carpentry workshops being a major factor in both the learning and its subsequent application. The only sustained criticisms were the lack of options for more advanced training and the absence of follow-up support to embed skills into practice.iii The efficacy of the Software Carpentry workshop model for the teaching of foundational computational skills has potential implications for the teaching of computer science and for the induction of research higher degree students into Australian universities.