Solutions to the problems of incremental validity testing in relationship science

Wang, Yilin Andre; Eastwick, Paul W.

doi:10.31234/osf.io/jm6xe

Cited by 4 publications

(5 citation statements)

References 73 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…But Wang and Eastwick (2020) described a third use of incremental validity testing, which may initially seem not to require causal theory, namely, the "isn't-it-just" argument. The isn't-it-just argument uses incremental validity to dispute a criticism that a new measure predicts only an outcome because it is a proxy for some already established predictor (e.g., "You found an effect of Machiavellianism on aggressive behavior, but isn't Machiavellianism just psychopathy?").…”

Section: Discussionmentioning

confidence: 99%

“…Controlling for a variable that shares variance with the outcome but not the predictor will decrease the amount of residual variance in the outcome, which, in turn, lowers the standard error of the estimated regression coefficient and increases power (Cohen et al, 2003). Control variables are also sometimes used to establish that a new measure is uniquely predictive beyond some already established measure (Wang & Eastwick, 2020; we discuss this practice later in the paper). Researchers may also be interested in using the partial regression coefficients to describe partial associations rather than using them to explain psychological processes.…”

Section: Linear Regression and Statistical Controlmentioning

confidence: 99%

See 1 more Smart Citation

Statistical Control Requires Causal Justification

Wysocki

Lawson

Rhemtulla

2022

Advances in Methods and Practices in Psychological Science

114

View full text Add to dashboard Cite

It is common practice in correlational or quasiexperimental studies to use statistical control to remove confounding effects from a regression coefficient. Controlling for relevant confounders can debias the estimated causal effect of a predictor on an outcome; that is, it can bring the estimated regression coefficient closer to the value of the true causal effect. But statistical control works only under ideal circumstances. When the selected control variables are inappropriate, controlling can result in estimates that are more biased than uncontrolled estimates. Despite the ubiquity of statistical control in published regression analyses and the consequences of controlling for inappropriate third variables, the selection of control variables is rarely explicitly justified in print. We argue that to carefully select appropriate control variables, researchers must propose and defend a causal structure that includes the outcome, predictors, and plausible confounders. We underscore the importance of causality when selecting control variables by demonstrating how regression coefficients are affected by controlling for appropriate and inappropriate variables. Finally, we provide practical recommendations for applied researchers who wish to use statistical control.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Linear Regression and Statistical Controlmentioning

confidence: 99%

Statistical Control Requires Causal Justification

Wysocki

Lawson

Rhemtulla

2022

Advances in Methods and Practices in Psychological Science

114

View full text Add to dashboard Cite

show abstract

“…In other words, we planned to calibrate our confidence in our results based on both the extent to which focal p-values reached significance and on the extent to which similar patterns of effect sizes emerged across different analytic approaches. We decided to focus primarily on the effect sizes and p-values from the SEM approach because estimates from latent variable models tend to be more accurate (less biased) than those from observed variable approaches and because an SEM approach helps avoid type I error inflation in this multivariate context (Ledgerwood & Shrout, 2011;Wang & Eastwick, 2020). At the same time, any one estimate from SEM analyses using latent variables can be quite far from the true population parameter (Ledgerwood & Shrout, 2011), so looking for consistent patterns across multiple analytic approaches can be informative.…”

Section: Resultsmentioning

confidence: 99%

Summarized Attribute Preferences Have Unique Antecedents and Consequences

Frost¹,

Wang²,

Eastwick³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

People have ideas about the attributes (i.e., traits or characteristics that vary along a dimension) that they like in others (e.g., “I like intelligence in a romantic partner”), and these ideas about liking are called summarized attribute preferences (Ledgerwood et al., 2018). But where do summarized preferences come from, and what do they predict? Across four studies, we examined how people form summarized attribute preferences and whether they predict situation selection. We showed participants a series of photographs of faces and assessed both their experienced liking for an attribute (or functional attribute preference) as well as their inference about how much they liked the attribute in the abstract (their summarized attribute preference). Our results suggest that summarized attribute preferences—despite being (weakly) grounded in functional attribute preferences—were affected by incidental aspects of the context in which people learn about them (i.e., the overall likeability of the pool of faces). Furthermore, we observed a double dissociation in the predictive validity of summarized and functional attribute preferences: Whereas summarized attribute preferences predicted situation selection at a distance (e.g., whether to join a new dating website based on a description of it), functional attribute preferences predicted situation selection with experience (e.g., whether to join a new dating website after sampling it). We discuss theoretical and methodological implications for the interdisciplinary science of human evaluation.

show abstract

“…In the pilot study, we will first examine whether status anxiety differs from another organizational attitude, job insecurity, using confirmatory factor analyses. That is, evidence beyond a test of predictive validity (e.g., with each variable explaining unique variance in job satisfaction) would more strongly bolster our assertion that these constructs are separate (Clark and Watson, 1995;Flake et al, 2017;Wang and Eastwick, 2020). After, in our main study, we will test whether status anxiety relates to job satisfaction above and beyond the discussed relevant factors.…”

Section: Overview Of Present Researchmentioning

confidence: 88%

Concerned Whether You’ll Make It in Life? Status Anxiety Uniquely Explains Job Satisfaction

Keshabyan

Day

2020

Front. Psychol.

View full text Add to dashboard Cite

Ever feel concerned that you may not achieve your career goals or feel worried about where your life is going? Such examples may reflect the experience of status anxiety, that is, concerns that one may be stuck or not able to move up in life, or worries that one may be too low in standing compared to society's standards. Status anxiety is believed to be exacerbated by economic inequality and negatively affect wellbeing. While job satisfaction is an important determinant of well-being, no research has examined whether status anxiety can also help explain people's satisfaction with their jobs. We tested whether status anxiety differs from other organizational constructs and uniquely relates to job satisfaction among full-time working adults. In a pilot study, we found that status anxiety is separate from the concept of job insecurity (e.g., perceived threat of job loss). Results of our main study also indicated that higher status anxiety significantly predicted lower job satisfaction beyond several other indicators of organizational attitudes (job insecurity, occupational self-efficacy, distributive, procedural, and interactional justice), as well as the tendency to seek status and several background factors (e.g., income, education, perceived socioeconomic status). We discuss the unique role of status anxiety in job satisfaction and the implications of this research to our understanding of status concerns, as well as organizational attitudes and policies.

show abstract

Solutions to the problems of incremental validity testing in relationship science

Cited by 4 publications

References 73 publications

Statistical Control Requires Causal Justification

Statistical Control Requires Causal Justification

Summarized Attribute Preferences Have Unique Antecedents and Consequences

Concerned Whether You’ll Make It in Life? Status Anxiety Uniquely Explains Job Satisfaction

Contact Info

Product

Resources

About