The challenges and benefits of analyzing feedback comments in surveys: Lessons from a cross-national online survey of small-scale cannabis growers

Abstract

It is common practice in survey questionnaires to include a general open and non-directive feedback question at the end, but the analysis of this type of data is rarely discussed in the methodological literature. While these open-ended comments can be useful, most researchers fail to report on this issue. The aim of this article is to illustrate and reflect upon the benefits and challenges of analyzing responses to open-ended feedback questions. The article describes the experiences of coding and analyzing data generated through a feedback question at the end of an international online survey with small-scale cannabis cultivators carried out by the Global Cannabis Cultivation Research Consortium. After describing the design and dataset of the web survey, the analytical approach and coding frame are presented. The analytical strategies chosen in this study illustrate the diversity and complexity of feedback comments which pose methodological challenges to researchers wishing to use them for data analyses. In this article, three types of feedback comments (political/policy comments, general comments of positive and negative appreciation, and methodological comments) are used to illustrate the difficulties and advantages of analyzing this type of data. The advantages of analyzing feedback comments are well known, but they seem to be rarely exploited. General feedback questions at the end of surveys are typically non-directive. If researchers want to use these data for research and analyses, they need a clear strategy. They ought to give enough thought to why they are including this type of question, and develop an analytical strategy at the design stage of the study.

Frequency, Stressfulness and Type of Ethically Challenging Situations Encountered by Veterinary Team Members During the COVID-19 Pandemic

“…The final question asked participants "is there anything else you would like to add about your experience with ethically challenging situations since the advent of COVID-19?" This question was included to act as a safety net, to facilitate identification of pertinent issues that were not addressed in the preceding questions (16). There were no restrictions on the length of answers.…”

Hidden and uninterested populations: Methodological insights and unresolved issues from the study of Cannabis Social Clubs

“…Previous cannabis studies (in particular, among cannabis growers) highlighted the importance of engaging with potential participants through websites and online forums, as well as through social media (e.g. Facebook), (Decorte et al, 2019; Barratt et al, 2015; Potter, 2017). For some of our CSC studies, we created Facebook pages or websites (S2, S3, S8).…”

“…In both online and offline descriptions of the studies, we were careful to avoid stigmatizing language, narrow labels, and explicitly clarified that the research team did not hold any negative preconceptions of cannabis use or cultivation, of CSCs, their members, or their activities. Similarly, the literature in this area has highlighted the need to make the recruitment process less intimidating to further enhance the likelihood that potential participants accede to take part in the research projects (Decorte et al, 2019; Ellard-Gray et al, 2015; Sandberg and Copes, 2013). On one occasion, however (S3), the way we described the research project in the participant information sheet was considered as potentially leading by the ethics committee reviewing our application.…”

“…Participation in research projects was used by some groups as a way of legitimizing their activities; by publicizing their “collaboration” with the University with which the research team was affiliated, emphasizing that public funding had been allocated to study their associations/movement, or even mobilizing some of our publications in the context of court cases. In S3, for instance, some of the participants expressed also appreciation of the research project in the final feedback question of the survey, writing “thank you for the work you do” (see also Decorte et al, 2019).…”

Research in fractured digital spaces

“…But it also means we have to consider whether we should operate with a gold standard of research and what that might be. Basing informed consent in a discussion about the purpose of the research widens the conversation about research and expertise, where the research population is invited to be part of a dialogue about their experiences and practices (Decorte et al, 2019). One aspect is practical.…”