Toward a model of grammaticality judgments

Abstract

This paper presents three experiments that investigate the relationship between gradient and binary judgments of grammaticality. In the first two experiments, two different groups of participants judged sentences by the method of magnitude estimation and by the method of speeded grammaticality judgments in a single session. The two experiments involved identical sentence materials but they differed in the order in which the two procedures were applied. The results show a high correlation between the magnitude estimation data and the speeded grammaticality judgments data, both within a session and across the two sessions. The third experiment was a questionnaire study in which participants judged the same sentences as either grammatical or ungrammatical without time pressure. This experiment yielded results quite similar to those of the other two experiments. Thus gradient and binary judgments both provide valuable and reliable sources for linguistic theory when assessed in an experimentally controlled way. We present a model based on Signal Detection Theory which specifies how gradient grammaticality scores are mapped to binary grammaticality judgments. Finally, we compare our experimental results to existing corpus data in order to inquire into the relationship between grammaticality and frequency of usage.

A comparison of informal and formal acceptability judgments using a random sample from Linguistic Inquiry 2001–2010

“…This development means that there are two methods for collecting acceptability judgments currently in widespread use in the field of syntax: the relatively informal traditional methods that have largely established the foundation of the field for the past 60 years (henceforth informal methods), and the more formal experimental methods that have been gaining popularity over the past 15 years (henceforth formal methods). This methodological dichotomy has led a number of researchers to ask which method is empirically superior (e.g., Bard et al, 1996;Keller, 2000;Edelman and Christiansen, 2003;Phillips and Lasnik, 2003;Featherston, 2005aFeatherston, , 2005bFeatherston, , 2007Featherston, , 2008Featherston, , 2009Ferreira, 2005;Sorace and Keller, 2005;Wasow and Arnold, 2005;den Dikken et al, 2007;Alexopoulou and Keller, 2007;Bornkessel-Schlesewsky and Schlesewsky, 2007;Fanselow, 2007;Grewendorf, 2007;Haider, 2007;Newmeyer, 2007;Sprouse, 2007;Culbertson and Gross, 2009;Myers, 2009aMyers, , 2009bPhillips, 2010;Bader and Häussler, 2010;Dąbrowska, 2010;Gibson and Fedorenko, 2010;Culicover and Jackendoff, 2010;Gross and Culberton, 2011;Sprouse, 2011b;Weskott and Fanselow, 2011;Gibson et al, 2011;Almeida, 2012, 2013;Gibson and Fedorenko, 2013). Our goal in this paper is to substantially increase the empirical basis of this line of research by comparing the results of informal and formal methods for a very large and random s...…”

Testing Japanese loanword devoicing: Addressing task effects

“…This gradient pattern is observed even when the participants use a binary yes/no method (see also Bader and Mäussler 2010;Coleman and Pierrehumbert 1997;Dankovičová et al 1998;Frisch et al 2004 for similar results). The current experiments thus show that gradient judgment patterns do not necessarily arise because many experiments in the past has used a rating scale; i.e.…”

“…Therefore, the second aim of this paper is to test whether the gradient results that Kawahara (2011aKawahara ( , 2011b obtained can be replicated using a binary yes/no task. Some previous studies (Bader and Mäussler 2010;Coleman and Pierrehumbert, 1997;Dankovičová et al, 1998;Frisch et al, 2004) raised similar issues and found gradient results using a binary yes/no format. The current study thus builds on them and aims to address the gradient nature of phonological judgments in the case of Japanese loanword devoicing.…”

Backward binding as a psych effect: A binding illusion?

“…7 Participants were instructed to make a binary choice: acceptable vs. non-acceptable. Previous studies show that scalar and binary acceptability measurements are equally informative for our purposes (Bader and Häussler 2010b;Weskott and Fanselow 2011 The fact that the coindexed reading of the marked material is not enforced may introduce a bias towards judging sentences as acceptable, but not with the intended reading. This potential bias is not confounded with the conditions of interest, i.e., it is expected to be constant across conditions.…”