Hypothetical case replacement can be used to quantify the robustness of trial results

Frank, Kenneth A.; Lin, Qinyun; Maroulis, Spiro; Mueller, Anna S.; Xu, Ran; Rosenberg, Joshua M.; Hayter, Christopher S.; Mahmoud, Ramy; Kolak, Marynia; Dietz, Thomas; Zhang, Lixin

doi:10.1016/j.jclinepi.2021.01.025

Cited by 26 publications

(12 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Scholars have also endeavored to build on the ITCV in order to develop a similar technique with identical inputs that works to address situations when the ITCV is less appropriate (we discuss the limitations of the ITCV in the coming sections). Although management scholars have referred to such an interpretation as the ITCV because the inputs for the calculations are identical to those for the correlation-based value, Frank et al (forthcoming) refer to this approach as the robustness of inference to replacement (RIR). The RIR provides insight into the percentage of a parameter estimate that would need to be biased in order to invalidate causal inference (Xu & Frank, forthcoming).…”

Section: Itcvmentioning

confidence: 99%

See 1 more Smart Citation

Omitted Variable Bias: Examining Management Research With the Impact Threshold of a Confounding Variable (ITCV)

Busenbark

Yoon²,

Gamache³

et al. 2021

Journal of Management

159

View full text Add to dashboard Cite

Management research increasingly recognizes omitted variables as a primary source of endogeneity that can induce bias in empirical estimation. Methodological scholarship on the topic overwhelmingly advocates for empirical researchers to employ two-stage instrumental variable modeling, a recommendation we approach with trepidation given the challenges associated with this analytic procedure. Over the course of two studies, we leverage a statistical technique called the impact threshold of a confounding variable (ITCV) to better conceptualize what types of omitted variables might actually bias causal inference and whether they have appeared to do so in published management research. In Study 1, we apply the ITCV to published studies and find that a majority of the causal inference is unlikely biased from omitted variables. In Study 2, we respecify an influential simulation on endogeneity and determine that only the most pervasive omitted variables appear to substantively impact causal inference. Our simulation also reveals that only the strongest instruments (perhaps unrealistically strong) attenuate bias in meaningful ways. Taken together, we offer guidelines for how scholars can conceptualize omitted variables in their research, provide a practical approach that balances the tradeoffs associated with instrumental variable models, and comprehensively describe how to implement the ITCV technique.

show abstract

Section: Itcvmentioning

confidence: 99%

“…The RIR value is calculated using identical inputs to the ITCV, but it deals with hypothetical changes to the underlying data rather than correlations (Frank et al, forthcoming; Xu et al, 2019). In other words, whereas the ITCV deals with correlations , the RIR quantifies how counterfactual changes in the data might invalidate causal inference.…”

Section: Rirmentioning

confidence: 99%

Omitted Variable Bias: Examining Management Research With the Impact Threshold of a Confounding Variable (ITCV)

Busenbark

Yoon²,

Gamache³

et al. 2021

Journal of Management

159

View full text Add to dashboard Cite

show abstract

“…Due to the observational nature of this study, unmeasured confounding factors may contribute to the estimated effect of submitting a postsecondary plan on expectational alignment. As a robustness check, I ran sensitivity analyses that quantify the level of uncertainty in my results (Frank et al 2021; Rosenberg et al 2018). Results from this analysis show that invalidating the effect of submitting a postsecondary plan in 9th or 11th grade on expectational alignment would require a substantial amount of unmeasured bias.…”

Section: Discussionmentioning

confidence: 99%

“…In the present study, the RIR identifies how many respondents who both submitted a postsecondary plan and expressed high-aligned expectations would need to be replaced with students for whom submitting a plan has no effect on their educational and occupational expectations to invalidate the estimate. The RIR is calculated by identifying the number of students who submit a plan and have aligned expectations that would need to be switched to uncertain expectations to nullify the effect (this metric is called Robustness of the Inference to Switches [RIS]), then dividing that number by the probability of having high-aligned rather than uncertain expectations (for further details, see Frank et al 2021).…”

Section: Sensitivity Analyses and Robustness Checksmentioning

confidence: 99%

“…Due to the relative novelty of this method, there is no established benchmark against which to judge these results (Frank et al 2021). However, one possible comparison for RIS is the difference in the unadjusted and adjusted probability of expressing high-aligned relative to uncertain expectations for students who submitted a postsecondary plan, that is, what percentage of respondents “switched” from high-aligned to uncertain expectations after adjusting with the covariates.…”

Section: Sensitivity Analyses and Robustness Checksmentioning

confidence: 99%

See 1 more Smart Citation

Planning for College and Careers: How Families and Schools Shape the Alignment of Postsecondary Expectations

Ahearn

2021

Sociol Educ

View full text Add to dashboard Cite

Students with aligned educational and occupational expectations have improved college and labor market outcomes. Despite extensive knowledge about the ways social background and school context contribute to educational expectations, less is known about the role of social intuitions in shaping expectational alignment. Drawing on data from the 2009 High School Longitudinal Study, I estimate the magnitude of socioeconomic inequality in alignment. I examine how differences in observed student characteristics contribute to, and whether school-based postsecondary planning initiatives mitigate, that inequality. Results from multinomial regression models show large socioeconomic differences in ninth-grade alignment, and I identify achievement, attitudes about college and careers, and relationships with significant others as contributors to those differences. Participation in postsecondary planning is associated with reduced uncertainty and increased alignment, but this relationship does not differ by social background, indicating that the examined college and career planning policies do little to address inequality in alignment.

show abstract

Public enemies? The differential effects of reputation and celebrity on corporate misconduct scandalization

Han,

Pollock,

Paruchuri

2024

Strategic Management Journal

View full text Add to dashboard Cite

Research SummaryWe explore misconduct scandalization's antecedents by focusing on the rational and emotional bases underlying reputation and celebrity, and considering how they can enhance or reduce the likelihood misconduct is scandalized as a function of the misconduct's objective and perceived severity. Specifically, we argue the quantifiable nature of objective misconduct severity enhances reputation's rational influence, but attenuates celebrity's emotion‐based appeal. Conversely, perceived misconduct severity reduces reputation's influence, while enhancing the media‐driven interest in celebrity firms' behaviors. Our findings based on corporate data breaches confirm that objective severity amplifies reputation's effect and attenuates celebrity's effect, while perceived severity amplifies celebrity's effect and attenuates reputation's effect. Our findings highlight the importance of social evaluations' sociocognitive content in understanding why only some misconduct becomes scandalized.Managerial SummaryCommitting misconduct is costly; having it scandalized is devastating. Yet little is known about how social evaluations influence why only some firms' misconduct is scandalized, beyond the vague notion that prominent firms' misconduct attracts media attention. We find that the rational and emotional bases of firms' evaluations matter. High reputation, based on the rational assessment of firms' capabilities, increases the likelihood of scandalization for objectively severe misconduct, and the influence of celebrity—originating from audiences' emotional resonance with firms' unconventional traits and behaviors—weakens as objective severity increases. Conversely, reputation's influence weakens, and celebrity's influence strengthens, as media “availability cascades” grow and increase perceived severity. In addition to providing a more realistic portrayal of media behavior, we offer insights into post‐misconduct communications and remedial actions.

show abstract

Hypothetical case replacement can be used to quantify the robustness of trial results

Cited by 26 publications

References 37 publications

Omitted Variable Bias: Examining Management Research With the Impact Threshold of a Confounding Variable (ITCV)

Omitted Variable Bias: Examining Management Research With the Impact Threshold of a Confounding Variable (ITCV)

Planning for College and Careers: How Families and Schools Shape the Alignment of Postsecondary Expectations

Public enemies? The differential effects of reputation and celebrity on corporate misconduct scandalization

Contact Info

Product

Resources

About