Reproducible research in linguistics: A position statement on data citation and attribution in our field

Abstract

This paper is a position statement on reproducible research in linguistics, including data citation and attribution, that represents the collective views of some 41 colleagues. Reproducibility can play a key role in increasing verification and accountability in linguistic research, and is a hallmark of social science research that is currently under-represented in our field. We believe that we need to take time as a discipline to clearly articulate our expectations for how linguistic data are managed, cited, and maintained for long-term access.

Replication in Second Language Research: Narrative and Systematic Reviews and Recommendations for the Field

“…This no doubt partially accounts for a history of inadequate reporting practices (e.g., as noted by Derrick, 2016;Han, 2016;Larson-Hall & Plonsky, 2015;Plonsky & Derrick, 2016), poor transparency of materials (Marsden & Mackey, 2014;Marsden et al, 2016;Marsden, Thompson, & Plonsky, in press), and very scarce availability of data (Larson-Hall & Plonsky, 2015;Larson-Hall, 2017;Plonsky, Egbert, & LaFlair, 2015). For an overview of these issues, see Marsden (in press); for discussions of similar challenges in linguistics, see Berez-Kroeker et al (2017), and in psychology, see Fecher, Friesike, and Hebing (2015), Lindsay (2017), and Wicherts, Borsboom, Kats, and Molenaar (2006). Indeed, aiming to address this situation, the Transparency and Openness Promotion (TOP) Guidelines (Nosek et al, 2015a(Nosek et al, , 2015b) encourage journals to incentivize/require their authors to make their materials and data transparent.…”

Introducing Registered Reports at Language Learning: Promoting Transparency, Replication, and a Synthetic Ethic in the Language Sciences

“…Second, small samples in human participant research may be unavoidable due to the limited resources available to many researchers. Indeed, in the language sciences, it may not be desirable or possible to insist on larger samples while we concurrently strive to expand participant demographics to hard‐to‐reach populations and to acknowledge the context sensitivity of language data (e.g., Berez‐Kroeker et al., ; Ortega, ).…”

“…Marsden et al. () found a near complete absence of such research (see also Berez‐Kroeker et al., ). A final impediment to replication research is a perception, at many levels, that replication research has low impact and prestige, although Marsden et al.…”

“…Second, small samples in human participant research may be unavoidable due to the limited resources available to many researchers. Indeed, in the language sciences, it may not be desirable or possible to insist on larger samples while we concurrently strive to expand participant demographics to hard-to-reach populations and to acknowledge the context sensitivity of language data (e.g., Berez-Kroeker et al, 2017;Ortega, 2013). Small data sets are less problematic under a more synthetic research ethic, where replications are synthesized within primary research that combines new data with previous data sets via meta-analysis (e.g., Ellis & Sagarra, 2011;Morgan-Short et al, 2018).…”

“…This prevents replication researchers from validating previous analyses to ascertain the reproducibility of the findings and blocks researchers from combining their own data with previous data sets. Marsden et al (2018) found a near complete absence of such research (see also Berez-Kroeker et al, 2017). A final impediment to replication research is a perception, at many levels, that replication research has low impact and prestige, although Marsden et al illustrate that, in fact, self-labeled replications have been relatively very well cited and published by journals with high impact factors.…”