Meta-Analyses in Survey Methodology: A Systematic Review

Čehovin, Gregor; Bošnjak, Michael; Manfreda, Katja Lozar

doi:10.1093/poq/nfy042

Cited by 7 publications

(7 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This paper complements previous meta-analyses that investigated differences in unit nonresponse between web and other survey modes (i.e., Daikeler et al, 2020; Lozar Manfreda et al, 2008). Further, it fills a research gap in survey methodology that was identified by a systematic review of meta-analyses in this field (Čehovin et al, 2018). While this paper found no overall difference in item nonresponse rate between web and other survey modes, two previous meta-analyses found that unit nonresponse in web surveys is 11–12 percentage points higher (on average) compared to other survey modes (Daikeler et al, 2020; Lozar Manfreda et al, 2008).…”

Section: Discussionmentioning

confidence: 99%

“…While several experimental studies have been conducted, item nonresponse in web surveys compared to other survey modes has not been investigated through meta-analysis (see Čehovin et al, 2018), which is a research gap that we aim to address in this paper. Specifically, the aim of this meta-analysis is to investigate the effect of survey mode (i.e., web vs. other survey modes) on differences in the mean rate of item nonresponse, which complements existing research on the issue of nonresponse.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Item Nonresponse in Web Versus Other Survey Modes: A Systematic Review and Meta-Analysis

Čehovin

Bošnjak

Manfreda

2022

Social Science Computer Review

Self Cite

View full text Add to dashboard Cite

Nonresponse is a fundamental issue in survey research, due to the trend of declining response rates across survey modes. This issue is particularly serious for web surveys: A recent meta-analysis found that unit nonresponse in web surveys is, on average, higher by 12 percentage points than in other modes ( Daikeler et al., 2020 ). Although the issue of unit nonresponse in web and other survey modes has been investigated in several meta-analyses, item nonresponse has not been examined meta-analytically, a gap addressed by this paper. Are web surveys at a disadvantage compared to other survey modes in terms of item nonresponse as well? To address this question, 13 eligible experimental manuscripts reporting 23 effect sizes were identified in a comprehensive literature search. Meta-analytic findings showed that there was no statistically significant difference in the average item nonresponse rate in web versus other survey modes. However, six moderator variables were found to statistically significantly affect the relation between the survey mode and the item nonresponse rate, namely, the target population, the number of contacts, the mode to which the web survey mode was compared, the survey sponsor, the age of the survey, and the baseline item nonresponse rate of the compared mode. The main practical implication is that while web and other survey modes differ in terms of unit nonresponse (on average), item nonresponse rates in web surveys are similar compared to other modes.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Item Nonresponse in Web Versus Other Survey Modes: A Systematic Review and Meta-Analysis

Čehovin

Bošnjak

Manfreda

2022

Social Science Computer Review

Self Cite

View full text Add to dashboard Cite

show abstract

“…Studies can and often are nested in more than one set of clusters at a time-that is, random factors may be crossed; to address that situation, cross-classified models can be used [91]. Many types of meta-analytical models have already been used to conduct research in the area of survey methodology, all of which either synthesize results from different papers or are based on self-calculated measures for selected survey programs [e.g., 10,37,40,41,48,59,[92][93][94][95][96][97][98].…”

Section: Meta-analysismentioning

confidence: 99%

“…I chose to include a moderator that for each observation simply counts how many surveys were done previously in this country and program. This study, as stated previously, is based on observational data-like many meta-analyses in survey methodology [98]. If one, however, would like to apply the categories of variables usually included in the causal inference models, then it should be noted that this study is focused chiefly on the effect of survey mode on nonresponse bias and that every candidate for the meta-regression moderator listed above should be treated as a confounder.…”

Section: Meta-analysismentioning

confidence: 99%

“…The best way of overcoming this issue is through experimental research, yet it is very costly and impossible to conduct on a scale such as was used in this study. Of 54 meta-analyses in survey methodology listed by Čehovin, Bos ˇnjak, and Lozar Manfreda, 28, more than half, used observational data [98]. Therefore, this limitation is real but very common in the field.…”

Section: Plos Onementioning

confidence: 99%

See 1 more Smart Citation

Survey mode and nonresponse bias: A meta-analysis based on the data from the international social survey programme waves 1996–2018 and the European social survey rounds 1 to 9

Rybak

2023

PLoS ONE

View full text Add to dashboard Cite

The constant increase in survey nonresponse and fieldwork costs are the reality of survey research. Together with other unpredictable events occurring in the world today, this increase poses a challenge: the necessity to accelerate a switch from face-to-face data collection to different modes, that have usually been considered to result in lower response rates. However, recent research has established that the simple response rate is a feeble measure of study quality. Therefore, this article aims to analyze the effect of survey characteristics, especially the survey mode, on the nonresponse bias. The bias measure used is the internal criteria first proposed by Sodeur and first applied by Kohler. The analysis is based on the survey documentation and results from the International Social Survey Programme waves 1996–2018 and the European Social Survey rounds 1 to 9. Random-effects three-level meta-regression models, based on data from countries from each inhabited continent, were created in order to estimate the impact of the survey mode or modes, sampling design, fieldwork experience, year of data collection, and response rate on the nonresponse bias indicator. Several ways of nesting observations within clusters were also proposed. The results suggest that using mail and some types of mixed-mode surveys were connected to lower nonresponse bias than using face-to-face mode surveys.

show abstract

From axial codes to Z‐scores: Charting the diversity and scope of 25 years of nonprofit marketing research published in International Journal of Nonprofit and Voluntary Sector Marketing

Waters

Auger

Bortree

2021

Journal of Philanthropy and Marketing

View full text Add to dashboard Cite

The International Journal of Nonprofit and Voluntary Sector Marketing published practitioner and scholarly research for 25 years before it was rebranded to the Journal of Philanthropy and Marketing. This research uses an emergent content analysis design to review all 629 research articles published in the first 25 volumes to document the field's marketing, theoretical, and methodological interests. This article provides important benchmark numbers to compare how the field evolves as the rebranded journal moves forward with a broader topical range.

show abstract

Meta-Analyses in Survey Methodology: A Systematic Review

Cited by 7 publications

References 24 publications

Item Nonresponse in Web Versus Other Survey Modes: A Systematic Review and Meta-Analysis

Item Nonresponse in Web Versus Other Survey Modes: A Systematic Review and Meta-Analysis

Survey mode and nonresponse bias: A meta-analysis based on the data from the international social survey programme waves 1996–2018 and the European social survey rounds 1 to 9

From axial codes to Z‐scores: Charting the diversity and scope of 25 years of nonprofit marketing research published in International Journal of Nonprofit and Voluntary Sector Marketing

Contact Info

Product

Resources

About