Switch to Web-based Surveys During Covid-19 Pandemic Left Out the Most Religious, Creating a False Impression of Rapid Religious Decline

Abstract

Religion appears to have taken a nosedive during the pandemic, including previously persistent forms of intense religion such as strong affiliation and biblical literalism. However, this apparent secularization is the result of mode effects. The gold standard General Social Survey switched to online rather than face-to-face interviews and the response rate plunged to 17%. Parallel analyses of GSS panel data demonstrate that this mode switch introduced substantial nonresponse bias. Illustratively, biblical literalism was almost 50% higher among those who declined to participate (36%) than those who participated in the online survey (25%). Rather than declining, intense religion persisted if not rose over time among those willing to participate in a push-to-web survey. The apparent decline was simply a result of disillusioned, distrusting, disinformed, disadvantaged, and disconnected people being much less likely to agree to participate in push-to-web surveys. Intense religion and other social phenomena are underrepresented and thereby underestimated in online surveys with substantial nonresponse, including those using population sampling methods. The trend in survey research toward these types of surveys could be expected to give a false impression of secularization and other social change going forward -- including making society look less disillusioned, distrusting, disinformed, disadvantaged, and disconnected than it is.