Open-Ended Survey Questions: Item Nonresponse Nightmare or Qualitative Data Dream?

Miller, Angie L.; Lambert, Amber D.

doi:10.29115/sp-2014-0024

Cited by 42 publications

(29 citation statements)

References 8 publications

(6 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…HNR referred to respondents who left answer boxes blank. Most of the studies reporting nonresponse in open-ended questions refer to HNR (e.g., Miller & Lambert, 2014). SNR meant that the respondents provided a nonresponse even though they wrote some text.…”

Section: Indicators and Analysismentioning

confidence: 99%

“…Some investigated the impact of survey stimuli on response quality, such as the size of answer boxes (Behr, Bandilla, Kaczmirek, & Braun, 2014;Christian & Dillman, 2004;Emde & Fuchs, 2012;Israel, 2010;Maloshonok & Terentev, 2016;Smyth, Dillman, Christian, & McBride, 2009), the number of answer boxes (Fuchs, 2009;Hofelich Mohr, Sell, & Lindsay, 2016;Keusch, 2014), the use of motivational sentences Oudejans & Christian, 2010;Smyth et al, 2009), clarification features (Metzler, Kunz, & Fuchs, 2015), examples (Tourangeau, Conrad, Couper, & Ye, 2014), the position of the open-ended question in the web survey (Miller & Lambert, 2014), and counters indicating the number of characters written (Emde & Fuchs, 2012). Other studies investigated the impact of respondents' characteristics such as age, gender, or education (Andrews, 2005;Barrios et al, 2011;Denscombe, 2008;Miller & Lambert, 2014;Smyth et al, 2009;Zuell, Menold, & Körber, 2015). To assess the response quality of open-ended questions in methodological studies such as listed above, but also in other research activities, researchers need to decide on indicators.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Using Apples and Oranges to Judge Quality? Selection of Appropriate Cross-National Indicators of Response Quality in Open-Ended Questions

Meitinger

Behr

Braun

2019

Social Science Computer Review

View full text Add to dashboard Cite

Methodological studies usually gauge response quality in narrative open-ended questions with the proportion of nonresponse, response length, response time, and number of themes mentioned by respondents. However, not all of these indicators may be comparable and appropriate for evaluating open-ended questions in a cross-national context. This study assesses the cross-national appropriateness of these indicators and their potential bias. For the analysis, we use data from two web surveys conducted in May 2014 with 2,685 respondents and in June 2014 with 2,689 respondents and compare responses from Germany, Great Britain, the United States, Mexico, and Spain. We assess open-ended responses for a variety of topics (e.g., national identity, gender attitudes, and citizenship) with these indicators and evaluate whether they arrive at similar or contradictory conclusions about response quality. We find that all indicators are potentially biased in a cross-national context due to linguistic and cultural reasons and that the bias differs in prevalence across topics. Therefore, we recommend using multiple indicators as well as items covering a range of topics when evaluating response quality in open-ended questions across countries.

show abstract

Section: Indicators and Analysismentioning

confidence: 99%

mentioning

confidence: 99%

Using Apples and Oranges to Judge Quality? Selection of Appropriate Cross-National Indicators of Response Quality in Open-Ended Questions

Meitinger

Behr

Braun

2019

Social Science Computer Review

View full text Add to dashboard Cite

show abstract

“…The fact that many participants were happy that their voices were heard may be linked to the general "participatory" approach chosen in our study. The methodological literature often suggests that completing open-ended response options requires a greater amount of time and mental effort than most closed-ended questions (Dillman, 2007), and that those with negative feelings about the study or the questionnaire are more likely to voice their opinions as comments, using them as a platform for their complaints (Miller and Dumford, 2014). This was not the case in our study.…”

Section: Discussionmentioning

confidence: 61%

“…In the methodological literature, open-ended comments from surveys have been described as a blessing and a curse for researchers (Dillman, 2007;Miller and Dumford, 2014;O'Cathain and Thomas, 2004). General open questions at the end of a survey may act as a safety net, helping researchers to identify issues not covered by closed questions even if the questionnaire was developed with considerable amounts of background research and piloting (Biemer et al, 2011).…”

Section: Introductionmentioning

confidence: 99%

The challenges and benefits of analyzing feedback comments in surveys: Lessons from a cross-national online survey of small-scale cannabis growers

Decorte

Malm

Sznitman

et al. 2019

Methodological Innovations

View full text Add to dashboard Cite

It is common practice in survey questionnaires to include a general open and non-directive feedback question at the end, but the analysis of this type of data is rarely discussed in the methodological literature. While these open-ended comments can be useful, most researchers fail to report on this issue. The aim of this article is to illustrate and reflect upon the benefits and challenges of analyzing responses to open-ended feedback questions. The article describes the experiences of coding and analyzing data generated through a feedback question at the end of an international online survey with small-scale cannabis cultivators carried out by the Global Cannabis Cultivation Research Consortium. After describing the design and dataset of the web survey, the analytical approach and coding frame are presented. The analytical strategies chosen in this study illustrate the diversity and complexity of feedback comments which pose methodological challenges to researchers wishing to use them for data analyses. In this article, three types of feedback comments (political/policy comments, general comments of positive and negative appreciation, and methodological comments) are used to illustrate the difficulties and advantages of analyzing this type of data. The advantages of analyzing feedback comments are well known, but they seem to be rarely exploited. General feedback questions at the end of surveys are typically non-directive. If researchers want to use these data for research and analyses, they need a clear strategy. They ought to give enough thought to why they are including this type of question, and develop an analytical strategy at the design stage of the study.

show abstract

“…We used the results of the code frequency and relational analyses to augment our interpretation of how codes related to each other and combined into themes, which facilitated our process of searching for themes, and defining and naming themes and sub-themes. Throughout data analysis, each researcher wrote "analytic memos" to record the process of coding, interpretations during analysis, and development of themes (Miles et al, 2014). We engaged in regular team meetings to discuss the ongoing analysis and emergent themes and revised themes to ensure that each theme was accurately reflected by the data and codes, and that themes (and their sub-themes) were distinct from each other.…”

Section: Data Analytic Approachmentioning

confidence: 99%

Racial Identity-Rooted Academic Motivation of First-Year African American Students Majoring in STEM at an HBCU

et al. 2021

View full text Add to dashboard Cite

The purpose the present study is to explore African American undergraduate students' perceptions of their experiences and academic motivation within a Historically Black College or University (HBCU) learning environment. As part of a larger study, we collected 212 open-ended survey responses from first year students in STEM majors about how the HBCU context shapes their academic motivation. We used semantic thematic data analysis and found three major themes and corresponding sub themes that were salient in the development of students' academic motivation: place (institutional climate, HBCU mission and tradition, and absence of marginalization); pedagogy (culturally relevant pedagogy, positive faculty-student relationships, African American curriculum and instruction, racial socialization); and people (people “like me”; student, faculty and alumni models of high achieving African Americans). We discovered that HBCU institutional factors engendered academic motivation that is rooted in students' racial identity and suggest the construct of racial identity-rooted academic motivation. Given the important and unique realities of African American students that impact their educational experiences, engagement, identity development, and achievement in various types of school contexts, self and sociocultural variables must be included in research and theory on the motivational psychology of African American students. Implications for higher education practice and future research are discussed.

show abstract

Open-Ended Survey Questions: Item Nonresponse Nightmare or Qualitative Data Dream?

Cited by 42 publications

References 8 publications

Using Apples and Oranges to Judge Quality? Selection of Appropriate Cross-National Indicators of Response Quality in Open-Ended Questions

Using Apples and Oranges to Judge Quality? Selection of Appropriate Cross-National Indicators of Response Quality in Open-Ended Questions

The challenges and benefits of analyzing feedback comments in surveys: Lessons from a cross-national online survey of small-scale cannabis growers

Racial Identity-Rooted Academic Motivation of First-Year African American Students Majoring in STEM at an HBCU

Contact Info

Product

Resources

About