Crowdsourcing is the practice of getting ideas and solving problems using a large number of people on the Internet. It is gaining popularity for activities in the engineering design process ranging from concept generation to design evaluation. The outcomes of crowdsourcing contests depend on the decisions and actions of participants, which in turn depend on the nature of the problem and the contest. For effective use of crowdsourcing within engineering design, it is necessary to understand how the outcomes of crowdsourcing contests are affected by sponsor-related, contest-related, problem-related, and individual-related factors. To address this need, we employ existing game-theoretic models, empirical studies, and field data in a synergistic way using the theory of causal inference. The results suggest that participants' decisions to participate are negatively influenced by higher task complexity and lower reputation of sponsors. However, they are positively influenced by the number of prizes and higher allocation to prizes at higher levels. That is, an amount of money on any following prize generates higher participation than the same amount of money on the first prize. The contributions of the paper are: (a) a causal graph that encodes relationships among factors affecting crowdsourcing contests, derived from game-theoretic models and empirical studies, and (b) a quantification of the causal effects of these factors on the outcomes of GrabCAD, Cambridge, MA contests. The implications of these results on the design of future design crowdsourcing contests are discussed.

Engineering design involves information acquisition decisions such as selecting designs in the design space for testing, selecting information sources, and deciding when to stop design exploration. Existing literature has established normative models for these decisions, but there is lack of knowledge about how human designers make these decisions and which strategies they use. This knowledge is important for accurately modeling design decisions, identifying sources of inefficiencies, and improving the design process. Therefore, the primary objective in this study is to identify models that provide the best description of a designer’s information acquisition decisions when multiple information sources are present and the total budget is limited. We conduct a controlled human subject experiment with two independent variables: the amount of fixed budget and the monetary incentive proportional to the saved budget. By using the experimental observations, we perform Bayesian model comparison on various simple heuristic models and expected utility (EU)-based models. As expected, the subjects’ decisions are better represented by the heuristic models than the EU-based models. While the EU-based models result in better net payoff, the heuristic models used by the subjects generate better design performance. The net payoff using heuristic models is closer to the EU-based models in experimental treatments where the budget is low and there is incentive for saving the budget. This indicates the potential for nudging designers’ decisions toward maximizing the net payoff by setting the fixed budget at low values and providing monetary incentives proportional to saved budget.

No abstract

Designers make process-level decisions to (i) select designs for performance evaluation, (ii) select information source, and (iii) decide whether to stop design exploration. These decisions are influenced by problem-related factors, such as costs and uncertainty in information sources, and budget constraints for design evaluations. The objective of this paper is to analyze individuals’ strategies for making process-level decisions under the availability of noisy information sources of different cost and uncertainty, and limited budget. Our approach involves a) conducting a behavioral experiment with an engineering optimization task to collect data on subjects’ decision strategies, b) eliciting their decision strategies using a survey, and c) performing a descriptive analysis to compare elicited strategies and observations from the data. We observe that subjects use specific criteria such as fixed values of attributes, highest prediction of performance, highest uncertainty in performance, and attribute thresholds when making decisions of interest. When subjects have higher budget, they are less likely to evaluate points having highest prediction of performance, and more likely to evaluate points having highest uncertainty in performance. Further, subjects conduct expensive evaluations even when their decisions have not sufficiently converged to the region of maximum performance in the design space and improvements from additional cheap evaluations are large. The implications of the results in identifying deviations from optimal strategies and structuring decisions for further model development are discussed.

This paper analyzes participation behaviors in design crowdsourcing by modeling interactions between participants and design contests as a bipartite network. Such a network consists of two types of nodes, participant nodes and design contest nodes, and the links indicating participation decisions. The exponential random graph models (ERGMs) are utilized to test the interdependence between participants' decisions. ERGMs enable the utilization of different network configurations (e.g., stars and triangles) to characterize different forms of dependencies and to identify the factors that influence the link formation. A case study of an online design crowdsourcing platform is carried out. Our results indicate that designer, contest, incentive, and factors of dependent relations have significant effects on participation in online contests. The results reveal some unique features about the effects of incentives, e.g., the fraction of total prize allocated to the first prize negatively influences participation. Further, we observe that the contest popularity modeled by the alternating k-star network statistic has a significant influence on participation, whereas associations between participants modeled by the alternating two-path network statistic do not. These insights are useful to system designers for initiating effective crowdsourcing mechanisms to support product design and development. The approach is validated by applying the estimated ERGMs to predict participants' decisions and comparing with their actual decisions.