Biased Sampling of Early Users and the Direction of Startup Innovation

Abstract

Using data from a prominent online platform for launching new digital products, we document that the composition of the platform's 'beta testers' on the day a new product is launched has a systematic and persistent impact on success. Specifically, we use word embedding methods to classify products launched on this platform as more or less focused on the needs of female customers, and show that female-focused products launched on a typical day -when nine-in-ten users on the platform are men -experience 40% less growth and are 5 percentage points less likely to have an any users a year after launch. Using exogenous variation driven by the platform's daily newsletter, we find that that the product gender gap shrinks on days when women are more likely to engage with the platform. Conversely, entrepreneurs who happen to launch a female-focused product on an especially male-dominated day reduce their product development efforts by roughly 30% and are 4 percentage points less likely to raise venture funding. Overall, our findings suggest that sample bias can systematically corrupt signals of a startup's market potential, bias entrepreneurial strategy, and so lead to a dearth of innovations aimed at consumers who are underrepresented among early-users.

Judging foreign startups

“…For entrepreneurs, our findings suggest caution when acting on feedback from accelerators. While past studies show that accelerators, by providing signals on startup's quality to the entrepreneur, are an important source of learning (Cohen, Bingham, & Hallen, 2019;Howell, 2020;Lyons & Zhang, 2018;Yu, 2020), such signals may lead entrepreneurs astray when they originate from a nonrepresentative sample or from biased actors (Cao et al, 2021). In our case, judges' foreign discounting implies that the signals from accelerators may be distorted for firms from regions under-represented among accelerator judges.…”

“…That said, later rounds of evaluation, where there is likely an opportunity to use local references and networks, may still require localized capabilities to best pick which global startups are most promising. Crucially, however, any global approach depends on accelerators revising their processes to reduce the impact of bias-in this case foreign bias-that all too often enters the evaluation of diverse and heterogenous samples (Brooks, Huang, Kearney, & Murray, 2014;Cao et al, 2021).…”

“…Instead, entrepreneurship studies turn to other intermediate milestones to proxy early-stage companies' performance and quality. One common measure is financing from angel investors or venture capitalists (Cao, Koning, & Nanda, 2021;Howell, 2017;Yu, 2020). This is a common measure because these investors' decisions reflect both selection and treatment effects that should result in startups with financing having higher startup performance.…”

“…Another increasingly common indicator is user traction, reflecting how much visibility and use a startup is getting from customers and other gatekeepers. Website page visits are becoming a common indicator for the latter in entrepreneurship studies to proxy startup performance (Cao et al, 2021;Hallen et al, 2020;Koning, Hasan, & Chatterji, 2022).…”

Benevolent Sexism and the Gender Gap in Startup Evaluation

“…Startups founded by women tend to be judged less favorably than those founded by men (Bigelow et al, 2014; Lee & Huang, 2018). Indeed, evidence of gender bias in evaluations of startup quality and potential has been documented among a wide range of stakeholder groups, including investors (Kanze et al, 2020), accelerator selection committees (Dutt & Kaplan, 2018), job applicants (Campero & Kacperczyk, 2020), and early users (Cao et al, 2021). This is troubling given that perceptions of a startup’s viability can significantly impact the willingness of stakeholders to support it with much needed resources (Davidsson et al, 2021; Fisher et al, 2017).…”