The Priest, the Sex Worker, and the CEO: Measuring Motivation by Job Type

Abstract

This study uses latent semantic analysis (LSA) to explore how prevalent measures of motivation are interpreted across very diverse job types. Building on the Semantic Theory of Survey Response (STSR), we calculate "semantic compliance" as the degree to which an individual's responses follow a semantically predictable pattern. This allows us to examine how context, in the form of job type, influences respondent interpretations of items. In total, 399 respondents from 18 widely different job types (from CEOs through lawyers, priests and artists to sex workers and professional soldiers) self-rated their work motivation on eight commonly applied scales from research on motivation. A second sample served as an external evaluation panel (n = 30) and rated the 18 job types across eight job characteristics. Independent measures of the job types' salary levels were obtained from national statistics. The findings indicate that while job type predicts motivational score levels significantly, semantic compliance as moderated by job type job also predicts motivational score levels usually at a lesser but significant magnitude. Combined, semantic compliance and job type explained up to 41% of the differences in motional score levels. The variation in semantic compliance was also significantly related to job characteristics as rated by an external panel, and to national income levels. Our findings indicate that people in different contexts interpret items differently to a degree that substantially affects their score levels. We discuss how future measurements of motivation may improve by taking semantic compliance and the STSR perspective into consideration.

Ordering motivation and Likert scale ratings: When a numeric scale is not necessarily better

“…It seems we are faced with a well-known problem when the structure of attributes found at the inter-individual level is not reproduced at the intra-individual level (Feldman, 1995;Mischel and Shoda, 1998;Molenaar et al, 2003). The effect of individual interpretations of statements on ratings has already been shown (Arnulf et al, 2020); our results showed this effect on the necessary condition for ratings, transitivity, as well.…”

“…Besides, individual interpretations of statements were found to deviate from interpretations commonly expected; moreover, these individual interpretations interfered with ratings independently of the strength of the attribute measured. That is, if one understands a statement differently, his/her rating of this statement will reflect not only agreement/disagreement, but also some personal characteristics (Arnulf et al, 2020). Given that ordering attributes requires respondents to unambiguously understand and distinguish paired statements, any deviation of individual interpretation from expected one can intrude ordering and hamper transitivity.…”

Factors Influencing Depressive Symptoms in Middle-Aged South Korean Workers by Job Type: A Population-Based Study

“…Researchers have targeted these issues among specific samples (e.g., medical personnel and certain job groups [ 29 ]) or compared influences based on factors, such as employment hours [ 30 ] and gender [ 31 ]. A previous study on job type tended to focus on details such as motivation [ 32 ]. In other words, most of the existing studies have focused on occupational conditions rather than specific job types, such as those in the white-collar (including professionals), service, and labor (including agricultural, forestry, fishery, craftspeople, and simple laborers) industries.…”

MOWDOC: A Dataset of Documents From Taking the Measure of Work for Building a Latent Semantic Analysis Space

“…Recently, organizational researchers (e.g., Arnulf et al, 2014 , 2018 ) have been able to use latent semantic analysis (LSA) and semantic survey response theory (SSRT) to quantify the semantic similarity between and among scales, items, and survey responses. Latent semantic analysis is a computational model that assesses similarity in language where the similarity of any “given word (or series of words) is given by the context where this word is usually found” (Arnulf et al, 2020 , p. 4). Latent semantic analysis involves establishing a semantic space from a corpus of existing documents (e.g., journal articles, newspaper stories, item sets).…”

“…Patterns of semantic similarity resulting from LSA have accounted for a substantive amount of variability in how individuals respond to survey items that purport to measure (a) transformational leadership, motivation, and self-reported work outcomes (60–86%; Arnulf et al, 2014 ), (b) employee engagement and job satisfaction (25–69%; Nimon et al, 2016 ), and (c) perceptions of a trainee program, intrinsic motivation, and work outcomes (31–55%, Arnulf et al, 2019 ). It also appears that personality, demographics, professional training, and interest in the subject matter have an impact on the degree to which an individual's responses follow a semantically predictable pattern (Arnulf et al, 2018 ; Arnulf and Larsen, 2020 , Arnulf et al, 2020 ). While being able to objectively access the degree to which survey responses are impacted by semantics is a great step forward in survey research, such research is often conducted with LSA spaces that are not open and therefore not customizable except by those that have access to the body of text upon which the LSA space is built.…”