Promoting Open Science Practices: What Can (Should) Graduate Programs Do?

Abstract

This project site contains a pre-print for a commentary / forum piece. The abstract is as follow: In the field of Applied Linguistics, calls for increasing Open Science practices, with the aim of promoting more transparent, reproducible, and replicable science, are regularly heard from established researchers (e.g. Godfroid & Hui 2020; Loewen & Godfroid 2020; Marsden 2020; Marsden & Plonsky 2018). At the same time, perspectives from the next generation of researchers, graduate students, appear to be lacking. This forum piece takes the first steps towards including graduate students in the conversation by discussing what Open Science means to this community. We illustrate how students can take advantage of the movement and how programs can better prepare students to embrace Open Science practices early in their career. Specifically, we reflect on three Open Science practices, pre-registration, open data, and open materials, and call for a more systematic training and incentive structure for these practices in graduate programs and in hiring and promotion policies.

Estimating reliability for response-time difference measures: Toward a standardized, model-based approach

“…Moreover, we join previous calls for the sharing of reproducible analysis code (Hui & Huntley, 2021;In'nami et al, 2022). Analysis code can reveal the procedure one has undertaken to arrive at the reported reliability.…”

Open science: Considerations and issues for TESOL research

“…Although TESOL and applied linguistics data more broadly are arguably rarely as sensitive as those in clinical trials, openly sharing data, if done without due caution, may violate the privacy of participants and other members of their respective communities. To help protect participants' data, at a minimum, the following steps should be taken: (a) conducting OS training sessions for data holders (Hicks, 2023; see also Hui & Huntley, in press for a multi‐tiered approach to incorporating OS training in applied linguistics graduate programs), (b) publishing field‐specific decision trees on sharing research data (e.g., Zipper et al., 2019, for water science research), (c) submitting data management plans with the manuscripts (e.g., “a written data privacy and security statement as part of the submission process;” Zipper et al., 2019, p. 5207), (d) discussing with participants how their data will be shared and ensuring that they can make an informed decision even if the purpose of the study is unknown (Dennis et al., 2019), (e) using controlled or managed access for sensitive data that can be reidentified (Dyke et al., 2015; Maritsch et al., 2022), and (f) sharing simulated data if the original data cannot be shared (In'nami, Mizumoto, Plonsky, & Koizumi, 2022). The latter is a viable option if the data are being collected from vulnerable groups (e.g., minorities and children).…”

(Why) Are Open Research Practices the Future for the Study of Language Learning?

“…Naming, describing, annotating, and organizing such files takes time. Although the benefits of such open research practices are expected to positively impact the validity, reliability, and replicability of research, the impact of the time commitment on researchers cannot be ignored (LeBel et al., 2017), particularly for graduate students and early career researchers who have limited time for completing milestones upon which their careers depend (Allen & Mehler, 2019; Hui & Huntley, 2020).…”