Conjoint measurement undone

Trendler, Günter

doi:10.1177/0959354318788729

Cited by 25 publications

(32 citation statements)

References 77 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…From a metrological perspective, Uher (2018) shows us which epistemological and methodological aspects in most psychological studies are ignored, with a marked reduction of the validity of those studies as a consequence. Recently, Trendler (2019a) revisited an ongoing debate about the justified use of conjoined measurement in psychological research (see also Krantz and Wallsten, 2019; Michell, 2019; Trendler, 2019b).…”

Section: Introductionmentioning

confidence: 99%

Should We Say Goodbye to Latent Constructs to Overcome Replication Crisis or Should We Take Into Account Epistemological Considerations?

Hanfstingl

2019

Front. Psychol.

View full text Add to dashboard Cite

This paper discusses theoretical and epistemological problems concerning validity of psychological science in the context of latent constructs. I consider the use of latent constructs as one reason for the replicability crisis. At the moment, there exist different constructs describing the same psychological phenomena side by side, and different psychological phenomena that are reflected by the same latent construct. Hagger called them déjà-variables, which lead to a decreasing validity of measurements and inhibit a deeper understanding of psychological phenomena. To overcome this problem, I suggest a shift of theoretical and epistemological perspective on latent constructs. One main point is the explicit consideration of latent constructs as mental representations, which change objects and are changed by objects via assimilative and accommodative processes. The explicit orientation toward assimilation and accommodation allows the control of normally automatized processes that influence our understanding of psychological phenomena and their corresponding latent constructs. I argue that assimilation and accommodation are part of our research practice anyway and cause the mentioned problems. For example, taking a measurement is an assimilative process, and thus a high measurement error should lead to an increase of accommodative processes. Taking into account these considerations, I suggest consequences for research practices, for individual researchers and for the philosophy of science.

show abstract

Section: Introductionmentioning

confidence: 99%

Should We Say Goodbye to Latent Constructs to Overcome Replication Crisis or Should We Take Into Account Epistemological Considerations?

Hanfstingl

2019

Front. Psychol.

View full text Add to dashboard Cite

show abstract

“…Third, serious epistemological criticisms of the use of psychometric scaling models to measure cognitive constructs abound. For an overview of literature discussing psychometrics as an entirely or partly pathological science, see H. Johnson (1936), Michell (1997), Michell (2008), Trendler (2013), Trendler (2009), Humphry (2013), Mari et al (2017), Maul, Irribarra, and Wilson (2016), Briggs (2013), Vautier et al (2012) and E. Lacot, Afzali, and Vautier (2016), among others. In a nutshell, these criticisms relate to two characteristics of standardized tests.…”

Section: Present State-of-the-art Methods Follow the Standards For Edmentioning

confidence: 99%

Measuring Unobserved Variables in Educational Inequality Research: Mathematics and Language 'Abilities' in Early Childhood

Sánchez¹

2019

Preprint

View full text Add to dashboard Cite

Evidence of social inequalities in cognitive abilities in early childhood has been documented in many societies; however, three characteristics of the data used to measure cognitive constructs make it difficult to quantify inequalities across groups. First, a causal understanding of validity is not compatible with the standard validation framework, which forces researchers to think critically what it means to measure unobserved constructs. Second, test scores only provide ordinal information about individuals, they are not interval scales and require the use of suitable corresponding methods for their study. Third, measurement invariance, which causes measurement error, may make comparison of test scores across groups invalid. The paper explores these three data problems applied to standardized tests---one mathematics and two language assessments---taken by a cohort of German children. The paper proposes a comparative validation framework for researchers based on nonparametric psychometric models and the representational theory of measurement. This framework can help researchers to determine if data fit the assumptions of a measurement model, to check for various forms of measurement error, and to overcome potential issues. A comparison of competing statistical modeling alternatives reveals substantial differences: By conceptualizing ability as ordinal instead of interval and excluding items that do not fit the assumptions of measurement models, I find a reduction in effect sizes for typical covariates studied in social stratification research.

show abstract

“…But the properties of psychical phenomena in themselves, especially non-sensory ones (e.g., thoughts, emotions, and motivations), cannot be empirically added (concatenated) or derived from additive quantities. The possibility of their measurement was therefore rejected by the British Association’s for the Advancement of Science committee for quantitative methods (Ferguson et al, 1940; see also Kant, 1786/2016; Trendler, 2018). This led psychologists and social scientists to focus on relational models, operational theories and utility concepts (Michell, 1999; Finkelstein, 2003).…”

Section: Measurement and Quantification Across The Sciencesmentioning

confidence: 99%

Quantitative Data From Rating Scales: An Epistemological and Methodological Enquiry

Uher

2018

Front. Psychol.

115

View full text Add to dashboard Cite

Rating scales are popular methods for generating quantitative data directly by persons rather than automated technologies. But scholars increasingly challenge their foundations. This article contributes epistemological and methodological analyses of the processes involved in person-generated quantification. They are crucial for measurement because data analyses can reveal information about study phenomena only if relevant properties were encoded systematically in the data. The Transdisciplinary Philosophy-of-Science Paradigm for Research on Individuals (TPS-Paradigm) is applied to explore psychological and social-science concepts of measurement and quantification, including representational measurement theory, psychometric theories and their precursors in psychophysics. These are compared to theories from metrology specifying object-dependence of measurement processes and subject-independence of outcomes as key criteria, which allow tracing data to the instances measured and the ways they were quantified. Separate histories notwithstanding, the article’s basic premise is that general principles of scientific measurement and quantification should apply to all sciences. It elaborates principles by which these metrological criteria can be implemented also in psychology and social sciences, while considering their research objects’ peculiarities. Application of these principles is illustrated by quantifications of individual-specific behaviors (‘personality’). The demands rating methods impose on data-generating persons are deconstructed and compared with the demands involved in other quantitative methods (e.g., ethological observations). These analyses highlight problematic requirements for raters. Rating methods sufficiently specify neither the empirical study phenomena nor the symbolic systems used as data nor rules of assignment between them. Instead, pronounced individual differences in raters’ interpretation and use of items and scales indicate considerable subjectivity in data generation. Together with recoding scale categories into numbers, this introduces a twofold break in the traceability of rating data, compromising interpretability of findings. These insights question common reliability and validity concepts for ratings and provide novel explanations for replicability problems. Specifically, rating methods standardize only data formats but not the actual data generation. Measurement requires data generation processes to be adapted to the study phenomena’s properties and the measurement-executing persons’ abilities and interpretations, rather than to numerical outcome formats facilitating statistical analyses. Researchers must finally investigate how people actually generate ratings to specify the representational systems underlying rating data.

show abstract

Conjoint measurement undone

Cited by 25 publications

References 77 publications

Should We Say Goodbye to Latent Constructs to Overcome Replication Crisis or Should We Take Into Account Epistemological Considerations?

Should We Say Goodbye to Latent Constructs to Overcome Replication Crisis or Should We Take Into Account Epistemological Considerations?

Measuring Unobserved Variables in Educational Inequality Research: Mathematics and Language 'Abilities' in Early Childhood

Quantitative Data From Rating Scales: An Epistemological and Methodological Enquiry

Contact Info

Product

Resources

About