Temporal Issues in Replication: The Stability of Centrality-Based Advantage

Abstract

Abstract:The results of archival studies may depend on when researchers analyze data for at least two reasons: (1) databases change over time and (2) the sampling frame, in terms of the period covered, may reflect different environmental conditions. We examined these issues through the replication of Hochberg, Ljungqvist, and Lu's (2007) research on the centrality of venture capital firms and their performance. We demonstrate (1) that one can reproduce the results in the original article if one uses data downloaded at roughly the same time as the original researchers did, (2) that these results remain fairly robust to even a decade of database updating, but (3) that the results depend sensitively on the sampling frame. Centrality only has a positive relationship to fund performance during boom periods.Keywords: centrality; replication; status; venture capital M uch of the recent attention to the need for replication has focused on confirming the results of experiments, particularly in medicine and in psychology (e.g., Ioannidis 2005; Simmons, Nelson, and Simonsohn 2011). Replication often matters in these settings because the high cost of doing certain sorts of research restricts these experiments to small samples, raising the potential for false positives both through random chance and through the intentional and unintentional decisions of the researchers (Simmons et al. 2011).By contrast, replication in sociology and other nonexperimental social science research might seem less important. Research in these fields typically involves regressions run on large, archival datasets. These studies have plenty of statistical power, reducing both the importance of the choices made by researchers and the odds of misattributing noise to being a signal. The data also often appear relatively immutable. Replication, therefore, would seem to involve little more than understanding the sensitivity of the results to the construction of variables and to estimation choices (e.g., Breznau 2015; Hannon and Defina 2016).But archival studies have their own sources of vulnerability. Databases, particularly those regularly updated by government agencies or commercial vendors, change as additional data become available and as errors get corrected. Sampling methods and questions may change subtly across years. Social processes and market forces also evolve and drift over time, meaning that results that hold in one longitudinal sample may not in another that ostensibly covers the same population over a slightly different time frame.Our replication explores the importance of these temporal issues-when the researcher engaged in the research and the period that he or she studied-through the replication and extended analysis of one of the most highly cited articles relat-107

The Persistent Effect of Initial Success: Evidence from Venture Capital

“…Because this mechanism depends to some extent on the supply of capital exceeding the demand for it, at least for the most promising deals, it also implies that the returns to past success -and the prestige associated with it -should become more pronounced during periods when venture capital becomes plentiful. Indeed, consistent with this expectation, Shi et al (2017), exploring the temporal sensitivity of the results in Hochberg et al (2007),…”

Qual a Posição Das Mulheres Na Rede De Board Interlocking Do Brasil? Uma Análise Para O Período De 1997 a 2015

“…A centralidade de autovetor expressa o prestígio do indivíduo na rede (Podolny, 1993;Sauder et al, 2012). Dessa forma, podemos interpretar a centralidade média de autovetor mais elevada das mulheres como uma possível saliência destas (Shi, Sorenson, & Waguespack, 2017). Entretanto, a diferença entre homens e mulheres foi significativamente discrepante (vide Gráfico 11) somente no ano de 2003.…”

What Is Women’s Position in Brazil’s Board Interlocking Network? An Analysis Covering the Period From 1997 to 2015

“…Eigenvector centrality measures individual prestige in a network (Podolny, 1993;Sauder et al, 2012). Thus, we can interpret women's higher mean eigenvector centrality as a possible prominence of this group (Shi, Sorenson, & Waguespack, 2017). However, the difference between men and women was significantly discrepant (see Chart 11) only in 2003.…”