Semantic measures: Using natural language processing to measure, differentiate, and describe psychological constructs.

Abstract

Psychological constructs, such as emotions, thoughts, and attitudes are often measured by asking individuals to reply to questions using closed-ended numerical rating scales. However, when asking people about their state of mind in a natural context ("How are you?"), we receive open-ended answers using words ("Fine and happy!") and not closed-ended answers using numbers ("7") or categories ("A lot"). Nevertheless, to date it has been difficult to objectively quantify responses to open-ended questions. We develop an approach using open-ended questions in which the responses are analyzed using natural language processing (Latent Semantic Analyses). This approach of using open-ended, semantic questions is compared with traditional rating scales in nine studies (N = 92-854), including two different study paradigms. The first paradigm requires participants to describe psychological aspects of external stimuli (facial expressions) and the second paradigm involves asking participants to report their subjective well-being and mental health problems. The results demonstrate that the approach using semantic questions yields good statistical properties with competitive, or higher, validity and reliability compared with corresponding numerical rating scales. As these semantic measures are based on natural language and measure, differentiate, and describe psychological constructs, they have the potential of complementing and extending traditional rating scales. (PsycINFO Database Record

The big two dictionaries: Capturing agency and communion in natural language

“…The distance between two vectors estimates whether two dictionaries are similar or different. More specifically, the semantic similarity score between two dictionaries is calculated as the cosine of the angle between the two vectors representing the dictionaries (for details on the generation of semantic space and the calculation of semantic similarity, see for example Kjell, Kjell, Garcia, & Sikstr€ om, 2018;Garcia & Sikstr€ om, 2014;Karlsson, Sikstr€ om, & Willander, 2013). The cosine score can conceptually be thought of as containing both words that overlap with each other (a direct and large contribution to the similarity score) and words that share similarity in meanings and are used in similar contexts (indirect contribution to the similarity score).…”

Culture Blind Leadership Research: How Semantically Determined Survey Data May Fail to Detect Cultural Differences

“…Using digital algorithms for text analysis, previous studies have found that widely used constructs within the OB domain are in fact semantically determined (Arnulf et al, 2014a(Arnulf et al, , 2018aNimon et al, 2015;Kjell et al, 2019). Digital algorithms take texts as their input and can perform computations on their meanings, comparing and grouping text according to quantitative measures of similarity.…”

“…In recent years, though, the assumptions underlying measurements have come under renewed scrutiny. Some of the core psychometric criteria for construct validation are not capable of falsifying erroneous hypotheses, and the "measurements" may be measuring quite different entities from what they purport (Slaney and Racine, 2013;Mari et al, 2017;Maul, 2017;Arnulf et al, 2018b;Kjell et al, 2019).…”

Abbreviated Three-Item Versions of the Satisfaction with Life Scale and the Harmony in Life Scale Yield Improved Psychometric Properties

“…All three datasets were collected online, and participants were first informed about the study, that participation is voluntary, anonymous, and that they can withdraw at any time without giving a reason. For more detailed procedural information about the collection of Dataset 1 and 2 see Kjell et al (2016Kjell et al ( , 2018, respectively.…”

“…That is, "harmony encourages a holistic world view that incorporates a balanced and flexible approach to personal well-being that takes into account social and environmental contexts" (Kjell et al, 2016, p. 894). In accordance to these definitions, individuals describe their SWL with words such as happy, content, fulfilled, pleased and gratified; and their HIL with words such as peaceful, balanced, calm, unity and agreement (Kjell, Kjell, Garcia, & Sikstr€ om, 2018). Further, in a large cross-cultural investigation where individuals where allowed to freely describe what happiness is for them, the responses concerned both harmony and psychological balance (25% of the responses) as well as satisfaction (7% of the responses; Delle Fave, Brdar, Freire, Vella-Brodrick, & Wissing, 2011, see also similar results in Delle Fave et al, 2016.…”