Analysis of rating scales: A pervasive problem in bilingualism research and a solution with Bayesian ordinal models

Veríssimo, João

doi:10.31234/osf.io/4m2ap

Cited by 3 publications

(3 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A related issue concerns the fact that, in our analyses, self-rated second-language speaking proficiency was treated as a continuous rather than an ordinal variable (Veríssimo, 2021). Treating self-rated proficiency as ordinal may have been more appropriate, especially if the distribution of values of self-rated speaking proficiency was somewhat unusual.…”

Section: Discussionmentioning

confidence: 99%

Bilingualism and executive attention: Evidence from studies of proactive and reactive control.

Spinelli

Goldsmith

Lupker

et al. 2022

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

According to some accounts, the bilingual advantage is most pronounced in the domain of executive attention rather than inhibition and should therefore be more easily detected in conflict adaptation paradigms than in simple interference paradigms. We tested this idea using two conflict adaptation paradigms, one that elicits a listwide proportion-congruent effect and one that elicits an item-specific proportion-congruent effect. In both cases, the relevant finding is that congruency effects are reduced when the proportion of congruent to incongruent items is smaller. These effects are validated measures of proactive and reactive control, respectively, and are aspects of executive attention known to be associated with individual differences in working memory capacity. We reasoned that if bilingualism affects executive attention in a similar way as does working memory capacity, indices of proactive and reactive control should be comparably associated with continuous variation in language status and working memory capacity. In two experiments, we replicated previous findings that working memory capacity is associated with variation in congruency effects (suggesting greater reliance on proactive control). In contrast, language status had no consistent association with performance, save for a hint that bilingualism may be associated with greater reliance on reactive control. Thus, the bilingual advantage may exist, but not in proactive control or any other aspects of executive attention that have been proposed thus far.

show abstract

Section: Discussionmentioning

confidence: 99%

Bilingualism and executive attention: Evidence from studies of proactive and reactive control.

Spinelli

Goldsmith

Lupker

et al. 2022

Journal of Experimental Psychology: Learning, Memory, and Cogni

View full text Add to dashboard Cite

show abstract

“…Lists were created following a Latin square design. We analyzed the results using Bayesian ordinal regression (Veríssimo, 2021; for details, see Appendix B). As expected, the preambles constructed to be plausible received systematically higher ratings than the ones constructed to be implausible.…”

Section: Methodsmentioning

confidence: 99%

Semantic Attraction in Sentence Comprehension

Laurinavichyute

Malsburg

2022

Cognitive Science

View full text Add to dashboard Cite

Agreement attraction is a cross-linguistic phenomenon where a verb occasionally agrees not with its subject, as required by grammar, but instead with an unrelated noun ('The key to the cabinets were . . . '). Despite the clear violation of grammatical rules, comprehenders often rate these sentences as acceptable. Contenders for explaining agreement attraction fall into two broad classes: Morphosyntactic accounts specifically designed to explain agreement attraction, and more general sentence processing models, such as the Lewis and Vasishth model ( 2005), which explain attraction as a consequence of how linguistic structure is stored and accessed in content-addressable memory. In the present research, we disambiguate between these two classes by testing a surprising prediction made by the Lewis and Vasishth model but not by the morphosyntactic accounts, namely that attraction should not be limited to morphosyntax, but that semantic features of unrelated nouns equally induce attraction. A recent study by Cunnings and Sturt (2018) provided initial evidence that this may be the case. Here, we report three single-trial experiments in English (N 1 = 1, 072, N 2 = 1, 426, and N 3 = 2, 454) that compared semantic and agreement attraction and tested whether and how the two interact. All three experiments showed strong semantically induced attraction effects closely mirroring agreement attraction effects. We complement these results with computational simulations confirming that the Lewis and Vasishth model can faithfully reproduce the observed results. In sum, our findings suggest that attraction is a more general phenomenon than is commonly believed, and therefore favor more general sentence processing models such as the Lewis and Vasishth model.

show abstract

“…Finally, in my own contribution (Veríssimo, 2021), I apply Bayesian ordinal models to the analysis of rating scales (e.g., of grammatical acceptability or language proficiency). I first show how the commonly-used statistical methods suffer from important flaws, and then illustrate how ordinal models can provide more valid, accurate, and informative inferences about graded constructs such as language proficiency.…”

mentioning

confidence: 99%

BLC mini-series: New statistical approaches and research practices for bilingualism research

Veríssimo¹

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Editorial introduction to the mini-series on ‘New statistical approaches and research practices for bilingualism research’, now in press at Bilingualism: Language and Cognition. Features contributions on: open data and materials (Bolibaugh, Vanek, & Marsden, 2021), preregistration (Mertzen, Lago, & Vasishth, 2021), statistical power (Brysbaert, 2021), meta-analysis (Plonsky, Sudina, & Hu), generalized additive models (Miwa & Baayen, 2021), divergence onset analysis (Stone, Lago, & Schad, 2021), and Bayesian ordinal models (Veríssimo, 2021).

show abstract

Analysis of rating scales: A pervasive problem in bilingualism research and a solution with Bayesian ordinal models

Cited by 3 publications

References 15 publications

Bilingualism and executive attention: Evidence from studies of proactive and reactive control.

Bilingualism and executive attention: Evidence from studies of proactive and reactive control.

Semantic Attraction in Sentence Comprehension

BLC mini-series: New statistical approaches and research practices for bilingualism research

Contact Info

Product

Resources

About