Takeaways from the Special Issue on The Practice of Replication

Reed, W. Robert

doi:10.5018/economics-ejournal.ja.2019-13

Cited by 4 publications

(3 citation statements)

References 25 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The CSI has had a huge impact. It seems to be a standard reference when discussing the (low) credibility of social science research with observational data (see e.g., Damian, Meuleman, and van Oorschot 2019; Reed 2019; Young 2018). Many, especially in media articles and Internet discussions, even concluded that the social sciences may not be rigorous enough: Findings depend to a large extent on who did the research.…”

Section: Introductionmentioning

confidence: 99%

Has the Credibility of the Social Sciences Been Credibly Destroyed? Reanalyzing the “Many Analysts, One Data Set” Project

Auspurg

Brüderl

2021

Socius

View full text Add to dashboard Cite

In 2018, Silberzahn, Uhlmann, Nosek, and colleagues published an article in which 29 teams analyzed the same research question with the same data: Are soccer referees more likely to give red cards to players with dark skin tone than light skin tone? The results obtained by the teams differed extensively. Many concluded from this widely noted exercise that the social sciences are not rigorous enough to provide definitive answers. In this article, we investigate why results diverged so much. We argue that the main reason was an unclear research question: Teams differed in their interpretation of the research question and therefore used diverse research designs and model specifications. We show by reanalyzing the data that with a clear research question, a precise definition of the parameter of interest, and theory-guided causal reasoning, results vary only within a narrow range. The broad conclusion of our reanalysis is that social science research needs to be more precise in its “estimands” to become credible.

show abstract

Section: Introductionmentioning

confidence: 99%

Has the Credibility of the Social Sciences Been Credibly Destroyed? Reanalyzing the “Many Analysts, One Data Set” Project

Auspurg

Brüderl

2021

Socius

View full text Add to dashboard Cite

show abstract

“…But we also use Frederick's CRT as a measurement device. Following Reed (2019), our study can thus be regarded as an "extension replication" that asks the question of whether the external validity of the Chandler et al study extends to populations other than the one they investigated.…”

Section: Introductionmentioning

confidence: 99%

Experienced survey participants, biased responses? – The example of cognitive reflection

Grüner

Hirschauer

2022

Applied Economics Letters

View full text Add to dashboard Cite

The example of cognitive reflectionThe Cognitive Reflection Test (CRT) is used in behavioral studies to describe individuals as intuitive or reflective. In recent years, behavioral scientists have increasingly recruited participants on online platforms. Besides convenience, apt platform recruitment is believed to increase validity and facilitate more meaningful inferences towards the population of interest compared to studies based on students. But particularly with regard to CRT, there is an oftenunderestimated drawback: "experienced clickers" might systematically distort findings.

show abstract

“…; and (ii) Does the intensity of the diaspora impact differ by diasporas’ attributes? To answer the first research question, we conduct one of the six types of replication recommended by Reed (2017, 2019), namely: focus on the same subject population but use different data. Specifically, we examine whether the institutional development role of diasporas found in the earlier literature extends to modifications to the dataset, while targeting the same population of migrants in OECD host countries.…”

Section: Introductionmentioning

confidence: 99%

How Robust Is the Evidence on the Impact of Diasporas on Institutional Quality in Home Countries?

Tran¹,

Cameron²,

Poot³

2020

Kyklos

View full text Add to dashboard Cite

SummaryThis paper conducts a robustness analysis of the impact of diasporas on institutional quality in home countries. Using the Database on Immigrants in OECD Countries (DIOC), we attest that the home country institutional development role of diasporas found in the literature is robust to modifications in terms of the dataset, the measurement of diasporas and the instrumental variable procedures used in our cross‐sectional and panel analyses. The novelty of this paper is that we take the heterogeneity of diasporas into account in terms of their distribution across host countries, their duration of stay, and the level of development of their home countries. As in earlier literature, we find robust evidence that diasporas enhance institutional quality in home countries. We also find that the intensity of the diffusion of advanced institutions from developed host countries to home countries through the international migration channel is weaker with diasporas characterized by shorter average duration of stay and with diasporas from developing home countries. However, this diffusion effect is not significantly related to the distribution of diasporas across OECD host countries.

show abstract

Takeaways from the Special Issue on The Practice of Replication

Cited by 4 publications

References 25 publications

Has the Credibility of the Social Sciences Been Credibly Destroyed? Reanalyzing the “Many Analysts, One Data Set” Project

Has the Credibility of the Social Sciences Been Credibly Destroyed? Reanalyzing the “Many Analysts, One Data Set” Project

Experienced survey participants, biased responses? – The example of cognitive reflection

How Robust Is the Evidence on the Impact of Diasporas on Institutional Quality in Home Countries?

Contact Info

Product

Resources

About