The Effects of Methods on Results in Dialectology

Abstract

Although dialectology was among the first disciplines to use the survey as a research tool, dialectologists, unlike researchers in other social sciences, have done little work in assessing the effects of their survey methods on their results. This paper attempts to begin a dialog on the effects of methods on results in dialectology by comparing results from five surveys which overlap both in their geographic coverage and in some of the linguistic information they elicit. The surveys differ in their methods of administration (face-to-face interviews vs. telephone interviews), sample construction (purposive vs. random), and the kinds of data they elicit (observations of behavior vs. self-reports). Our comparison of the different surveys shows that while different modes of administration have little effect on results, even slight differences in survey populations affect results significantly, as does the kind of data elicited. Surprisingly, self-reports seem to reflect the linguistic behavior of a population for some features better than observations of behavior do. The effects of the type of sample used are not clear from this study, although random samples have the advantage of explicitly accounting for sampling error and allow for a wide range of inferential statistics that cannot be used with purposive samples. Finally, the comparison suggests that there is no single 'best' type of survey. Different research problems require different kinds of surveys. What is important is that samples not be constructed in haphazard ways and that we explicitly take into account the effects of our methods on our results.

needs+PAST PARTICIPLE in regional Englishes on Twitter

“…Production data of needs +PAST on Twitter in this study strongly confirm the conclusions of those researchers, and more broadly validate the methodologies they employed to study needs +PAST. While previous work has defended the practice of studying low‐frequency linguistic features via elicitation of conscious judgments from informants on pragmatic grounds (Murray & Simon, ; Murray et al., ) or on the basis of reliability within instruments (Benson, , ) and between instruments (Bailey et al., ; Chambers, ), the production data in this study validate those methods as having provided a basically accurate description of needs +PAST in actual language behavior. While Doyle () used Labov et al.…”

“…Benson (, p. 48; , p. 232) challenges that concerns over the validity of informant judgments are ‘largely anecdotal.’ She defends the use of acceptability judgments for studying low‐frequency features on the basis of consistency of responses within her own survey data (Benson, , p. 49), and points to Youmans () and Bailey, Wikle, and Tillery () as part of ‘a growing body of evidence [that] attests to the reliability of the data and the validity of the conclusions based on acceptability judgments of morpho‐syntactic features’ (Benson, , p. 49). Chambers () also shows that responses to questions in a fieldworker‐administered survey are highly similar to responses to a survey conducted by mail.…”

“…() over the problematic nature of survey conscious judgments about a feature of which people may not be conscious. Bailey, Wikle, and Tillery (, p. 57) do compare results from self‐reports against observed behavior, and conclude that ‘self‐reports might be more valid and reliable measures of linguistic behavior than linguists have supposed.’ However, their defense of self‐reports for low‐frequency grammatical features is based on the fact that they ‘are extremely difficult to elicit’ (Bailey et al., , p. 57). As such, ‘any survey of these features that relies on observed behavior will surely under‐report their occurrence’ (Bailey et al., , pp.…”

Investigating the Effect of Socio-Cognitive Salience and Speaker-Based Factors in Morpho-Syntactic Life-Span Change

“…The panel data analysis is beleaguered by a caveat that plagues the lion's share of longitudinal research, especially in the area of morpho-syntax: the reliance on low token counts (Sankoff 2004;Rickford & Price 2013). The findings reported here thus need to be confirmed on the basis of larger, independent trend or panel samples (which are extremely difficult to construct; see Cheshire 1982Cheshire , 2005Bailey, Wikle & Tillery 1997). Throughout the article, the significance of results will be reported via chi-square tests (χ 2 ).…”

Some Effects of Transcribers on Data in Dialectology

“…This paper builds on our earlier work on the effects of methods on results in dialectology and sociolinguistics (Bailey, Wikle, and Tillery 1997;Bailey and Tillery 1999); that work, in turn, builds on Labov's identification of the observer's paradox and its effects on data in sociolinguistics (1966,1972). Neither subdiscipline has a very extensive literature on the effects of methods.…”