The Role of Feedback in Dynamic Crowdsourcing Contests: A Structural Empirical Analysis

Abstract

In this paper, we empirically examine the impact of performance feedback on the outcome of crowdsourcing contests. We develop a dynamic structural model to capture the economic processes that drive contest participants’ behavior and estimate the model using a detailed data set about real online logo design contests. Our rich model captures key features of the crowdsourcing context, including a large participant pool; entries by new participants throughout the contest; exploitation (revision of previous submissions) and exploration (radically novel submissions) behaviors by contest incumbents; and the participants’ strategic choice among these entry, exploration, and exploitation decisions in a dynamic game. Using counterfactual simulations, we compare the outcome of crowdsourcing contests under alternative feedback disclosure policies and award levels. Our simulation results suggest that, despite its prevalence on many platforms, the full feedback policy (providing feedback throughout the contest) may not be optimal. The late feedback policy (providing feedback only in the second half of the contest) leads to a better overall contest outcome. This paper was accepted by Gabriel Weintraub, revenue management and market analytics department.

Submitting tentative solutions for platform feedback in crowdsourcing contests: breaking network closure with boundary spanning for team performance

“…Therefore, making tentative submissions for preliminary feedback is indeed helpful for the enhancement of team performance. This is consistent with the simulation results in a previous study that platform feedback helps contest teams improve crowdsourcing outcome (Jiang et al. , 2022).…”

“…Based on such feedback as preliminary scores and rankings, the team can understand how well it is doing and improve the work accordingly. Previously, it was found that final scores helps team members understand their strengths and weaknesses for skill enhancement in future contests (Jiang et al, 2022). This study further posits that team outreach for platform feedback during a contest is conducive to eventual outcome.…”

Do Dynamic Signals Affect High-Quality Solvers’ Participation Behavior? Evidence from the Crowdsourcing Platform

“…Scholars believe that static stock data cannot effectively reflect the changes in project design parameters over time. Therefore, the dynamics of the crowdsourcing contest process have received more attention [31]. Posts with timestamps show the knowledge-sharing trajectory of participants on the crowdsourcing work platform, making the knowledge-sharing trajectory a channel for platform participants to understand and influence each other.…”

“…However, the focus of crowdsourcing contests is to guide more solvers to participate and obtain more high-quality solutions. Therefore, Jiang et al [31] simulate multiple stages of the crowdsourcing competition by building a dynamic structural model, and explore the impact of feedback on the solver's entry into the competition, exploration and development of new solutions. Similarly, Sanyal and Ye [30] also posit that feedback is a tool to dynamically improve the efficiency of public competitions.…”

“…However, previous studies are based on static signals, without considering the variation in the signal over time and the difference in its impact. Scholars have considered the dynamic development process of crowdsourcing and explored the impact of feedback on dynamic performance from the perspective of the seeker [30,31]. However, what is revealed about the changes over time in the project itself is also worthy of attention.…”