Exploring qualitative training reactions: Individual and contextual influences on trainee commenting.

Abstract

Training reactions are the most common criteria used for training evaluation, and reaction measures often include opportunities for trainees to provide qualitative responses. Despite being widely used, qualitative training reactions are poorly understood. Recent trends suggest commenting is ubiquitous (e.g., tweets, texting, Facebook posts) and points to a currently untapped resource for understanding training reactions. In order to enhance the interpretation and use of this rich data source, this study explored commenting behavior and investigated 3 broad questions: who comments, under what conditions, and how do trainees comment? We explore both individual difference and contextual influences on commenting and characteristics of comments in 3 studies. Using multilevel modeling, we identified significant class-level variance in commenting in each of the 3 samples of trainees. Because commenting has only been considered at the individual level, our findings provide an important contribution to the literature. The shared experience of being in the same class appears to influence commenting in addition to individual differences, such as interest in the topic (Studies 1 and 2), satisfaction (Studies 2 and 3), and entity beliefs (Study 3). Furthermore, we demonstrated that item wording may have an impact on commenting (Study 3) and should be considered as a potential lever for training professionals to influence commenting behavior from trainees. Training professionals, particularly those who regularly administer training evaluation surveys, should be aware of nonresponse to open-ended items and how that may impact the information they collect, use, and present within their organizations.

Insufficient Effort Survey Responding: An Under‐Appreciated Problem in Work and Organisational Health Psychology Research

“…To this end, researchers may consider increasing rapport with their respondents by displaying a video clip of researchers reading the survey instructions (Ward, Meade, Gasperson, & Pond, ), issuing a benign note about the existence of IER detection tools (Huang et al, ), and creating a tie between participant incentives and data quality (see Huang et al, ). As noted, researchers have found that participants are more likely to provide comments when they have an interest in the topic (Harman et al, )—interest may not be able to be manipulated by researchers, but providing some information on the survey topic and the projected use of the response data may help. Future research in this area should include interventions to reduce IER in online surveys, such as real‐time feedback in online surveys for individuals who fail more than one IER item or go through survey pages in an unlikely rapid fashion.…”

“…In other words, more satisfied groups are more likely to respond to organisational surveys. Also, Harman et al () found that participants in a training program were more likely to provide comments (training reactions) when they had an interest in the topic. We speculate that this could extend to individuals having an interest in the survey topic being more likely to pay attention to items and display lower levels of IER.…”

“…A substantial literature examines this issue (see Rogelberg & Stanton, ), and has identified predictors of nonresponse (e.g. Fauth, Hattrup, Mueller, & Roberts, ; Harman, Ellington, Surface, & Thompson, ; Mueller, Voelkle, & Hattrup, ; LaRose & Tsai, ). Importantly, similar to CMV, nonresponse does not necessarily mean that nonresponse bias will occur (Rogelberg & Luong, ).…”

Preditores da aprendizagem e instrumentos de transferência de treinamento: um estudo com profissionais da saúde pública

“…No contexto da EAD a reação pode também ser avaliada qualitativamente, por meio de comentários em tweets, mensagens de texto e postagens no Facebook. A avaliação de reação qualitativa ainda é pouco explorada por estudos cintíficos, mas há evidências de que os comentários dos treinandos influenciam os comportamentos dos estudantes e isso pode predizer a aprendizagem e a efetividade do treinamento (Harman, Surface, Ellington, & Thompson, 2015).…”

Multilevel modeling issues and the measurement of stress is multilevel data

“…Research utilizing MLMs are common in IO psychology. A review of recent articles (2012late spring of 2015) in The Journal of Applied Psychology, The Academy of Management Journal, and Personnel Psychology (see Table 1) reveals studies across a wide variety of subjects, including creativity (e.g., Chang, Jia, Takeuchi, & Cai, 2014;Hirst, Van Knippenberg, Zhou, Quintane, & Zhu, 2015), training reactions (e.g., Harman, Ellington, Surface, & Thompson, 2014), culture and commitment (e.g., Fisher, 2014), groups and teams (e.g., Chang et al, 2014;Farmer, Van Dyne, & Kamdar, 2015;Gonzalez-Mule, DeGreest, McCormick, Seong, & Brown, 2014) and leadership (e.g., Chen, Zhu, & Zhou, 2015;Dong, Liao, Chuang, Zhou & Campbell, 2015;Farh & Chen, 2014;Liu, Wang, Chang, Shi, Zhou, & Shao, 2014).…”