Correlation with Time Explains the Relationship between Survey Nonresponse and Mass Polarization

Mellon, Jonathan; Prosser, Christopher

doi:10.1086/709433

Cited by 6 publications

(3 citation statements)

References 18 publications

(15 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Costs are increasing, response rates are declining (Keeter et al 2017), and there are growing concerns about inaccuracy (Kennedy et al 2018;Clinton et al 2021a. These patterns have raised questions about biases due to coverage and non-response (e.g., Cavari and Freedman 2022, though see Mellon and Prosser 2021) and have even led some to question the viability of polling itself (Meyer et al 2015;Keeter 2018).…”

Section: Introductionmentioning

confidence: 99%

Synthetic Replacements for Human Survey Data? The Perils of Large Language Models

Bisbee¹,

Clinton²,

Dorff³

et al. 2023

Preprint

View full text Add to dashboard Cite

Large Language Models (LLMs) offer new research possibilities for social scientists, but their potential as “synthetic data” is still largely unknown. In this note, we investigate the potential of using the popular closed-source LLM ChatGPT to measure human opinion. We show that although ChatGPT-generated opinions are similar to human opinion for some groups of US respondents, synthetic opinions also significantly exaggerate the extremity and certainty of partisan and social divisions. Responses from prompted “persona” profiles in ChatGPT produce measures of partisan and racial affective polarization that are seven times larger than the average opinion of humans who possess the same attributes as the prompted personas. Furthermore, synthetic data are artificially precise, with a standard deviation that is only 31% of the variation found in actual opinions among comparable humans. Because LLMs are proprietary, it is difficult to pinpoint the source of these biases, but our findings raise important questions about the appropriateness of replacing human opinion with synthetic responses generated by closed-source LLMs.

show abstract

Section: Introductionmentioning

confidence: 99%

Synthetic Replacements for Human Survey Data? The Perils of Large Language Models

Bisbee¹,

Clinton²,

Dorff³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Analyzing rich survey data from Pew, we demonstrated that as survey response declines, survey samples overrepresent politically engaged respondents who report more polarized views on several domestic issues. Using the same data and additional simulations, Mellon and Prosser (2021; hereafter MP) suggest that the relationship between survey response and nonresponse bias depends on the cause of low survey response, specifically, declining contact rates , associated with random polling mechanisms and caller ID features that offer screening abilities, or declining cooperation rates , associated mostly with personal preferences, knowledge, and interest in politics.…”

mentioning

confidence: 99%

Survey Nonresponse and Mass Polarization: The Consequences of Declining Contact and Cooperation Rates

Cavari¹,

Freedman

2022

Am Polit Sci Rev

View full text Add to dashboard Cite

Recent studies question whether declining response rates in survey data overstate the level of polarization of Americans. At issue are the sources of declining response rates—declining contact rates, associated mostly with random polling mechanisms, or declining cooperation rates, associated with personal preferences, knowledge, and interest in politics—and their differing effects on measures of polarization. Assessing 158 surveys (2004–2018), we show that declining cooperation is the primary source of declining response rates and that it leads to survey overrepresentation of people who are more engaged in politics. Analyzing individual responses to 1,223 policy questions in those surveys, we further show that, conditional on the policy area, this survey bias overestimates or underestimates the partisan divide among Americans. Our findings question the perceived strength of mass polarization and move forward the discussion about the effect of declining survey response on generalizations from survey data.

show abstract

“…Note that I also include a covariate to account for the passage of time, . This is important, as failing to account for secular changes that occur over time can confound one's estimates(Mellon and Prosser 2020;Woolridge 2012). Likewise, as this is a Bayesian model, I also include a prior distribution for each parameter.…”

mentioning

confidence: 99%

What's the Half-Life of the Economic Vote? (About a Year and a Half)

Bailey

2021

Preprint

View full text Add to dashboard Cite

Economic voting research assumes that voters focus their attention on the recent past. Yet testing this assumption is difficult and previous research remains inconclusive. To estimate voters' economic time frames, I specify a new model that relies on insights from the physical sciences. I show that voter myopia is real and that the economic vote is strongest when economic time frames are shortest.After around a year and a half, its effect falls by half. After seven years, it becomes practically equivalent to zero. This suggests that voters are less short-sighted than some past research suggests. Yet, there is still some cause for concern: voters' economic time frames remain short enough that governments may receive undeserved leeway for mistakes they make early in their tenure.

show abstract

Correlation with Time Explains the Relationship between Survey Nonresponse and Mass Polarization

Cited by 6 publications

References 18 publications

Synthetic Replacements for Human Survey Data? The Perils of Large Language Models

Synthetic Replacements for Human Survey Data? The Perils of Large Language Models

Survey Nonresponse and Mass Polarization: The Consequences of Declining Contact and Cooperation Rates

What's the Half-Life of the Economic Vote? (About a Year and a Half)

Contact Info

Product

Resources

About