Write‐in comments in employee surveys

Abstract

Purpose -Items that ask the respondent for a comment in his/her own words are becoming increasingly popular in online employee surveys, but research on such comments is scarce. The purpose of this paper is to analyze, theoretically and empirically, what kind of comments are generated in employee surveys by respondents who differ in terms of job satisfaction and commitment. Design/methodology/approach -The data studied here are from an online employee survey conducted in 2004 in a multinational IT organization with a German headquarter. Some 24,000 employees generated about 75,000 comments focusing on 15 topic fields. The comments were additionally coded on their tone, using a computerized dictionary approach especially developed for this purpose. Frequencies, wordiness, and the tone of different types of comments are measured. The statistical relationship of comments to job satisfaction and to organizational commitment is analyzed. Findings -Some 40 per cent of the respondents provide comments. Most comments have a negative tone. Negative comments are wordier than positive ones. The likelihood of writing comments is inversely related to the respondents' job satisfaction and to their organizational commitment. Dissatisfied employees and employees with low commitment also write more negative and wordier comments. Practical implications -The study sets benchmarks on what to expect when using open-ended comment fields in employee surveys. A methodology for analyzing huge text files with respect to their tone is presented. Originality/value -This is the first paper that investigates a realistically large data set of comments. It also shows how to use computer methods to relate frequencies, wordiness, and tone of comments to standard variables such as job satisfaction or commitment.

2016 Assessment of the Civilian Acquisition Workforce Personnel Demonstration Project

“…Finally, the write-in survey responses could not be adjusted for response bias. Academic studies have shown that there is a general tendency toward negativity in openended responses to employee surveys (Andrews, 2005;Borg and Zuell, 2012;Poncheri et al, 2008). Unfortunately, we could not correct for this issue.…”

“…In particular, we could not weight the data as we did for the quantitative survey responses. Moreover, given the numerous and diverse ways in which respondents and nonrespondents differed, we could not accurately assess the nature and extent of their bias beyond the general bias toward negativity present in write-in comments on employee surveys (Andrews, 2005;Borg and Zuell, 2012;Poncheri et al, 2008). We observed this negativity bias in our coding of AcqDemo survey write-ins: Regardless of the topic, responses with a negative tone were far more prevalent than those expressing positive sentiments.…”

“…However, we do note that there is limited evidence in the academic literature of an overall negativity bias associated with write-in responses to employee surveys. Specifically, dissatisfied employees are more likely to provide responses, and their responses tend to be longer (Andrews, 2005;Borg Poncheri et al, 2008). In addition, Poncheri et al (2008) and Bord and Zuell (2012) both found that the tone of open-ended comments in such surveys tends to be disproportionately negative.…”

The promise of open survey questions—The validation of text-based job satisfaction measures

“…Sentiment analysis appears to be a suitable method for the creation of a text-based measure. As suggested by several empirical studies [e.g., 34–38], the sentiment found in texts seems to be a natural manifestation of the pleasant and unpleasant emotions, beliefs and cognitions employees have–factors that jointly constitute job satisfaction [32, e.g., 39,40].…”

Using correspondence analysis to evaluate consumer terminology and understand the effects of smoking method and type of wood on the sensory perception of smoked meat

“…In addition, some researchers reported that they found a negative bias in the answering behavior to open questions about job satisfaction in employee surveys (Poncheri, Lindberg, Thompson, & Surface, 2008). Unsatisfied respondents answered more often and with longer answers to openended questions about situations eliciting negative feelings than satisfied respondents (Borg & Zuell, 2012;Poncheri et al, 2008). Open-ended questions have been underutilized or not exploited to their full potential in the past, due to laborious and time-consuming data processing.…”