Evaluating the Applicability of Peer-Designed Agents in Mechanisms Evaluation

2015

jair

Self Cite

This paper considers the problem of cooperation between self-interested agents in acquiring better information regarding the nature of the different options and opportunities available to them. By sharing individual findings with others, the agents can potentially achieve a substantial improvement in overall and individual expected benefits. Unfortunately, it is well known that with self-interested agents equilibrium considerations often dictate solutions that are far from the fully cooperative ones, hence the agents do not manage to fully exploit the potential benefits encapsulated in such cooperation. In this paper we introduce, analyze and demonstrate the benefit of five methods aiming to improve cooperative information gathering. Common to all five that they constrain and limit the information sharing process. Nevertheless, the decrease in benefit due to the limited sharing is outweighed by the resulting substantial improvement in the equilibrium individual information gathering strategies. The equilibrium analysis given in the paper, which, in itself is an important contribution to the study of cooperation between self-interested agents, enables demonstrating that for a wide range of settings an improved individual expected benefit is achieved for all agents when applying each of the five methods.

Section: Related Workmentioning

confidence: 99%

Constraining Information Sharing to Improve Cooperative Information Gathering

Rochlin

2015

jair

Self Cite

“…The students were requested to send us their strategy seven days after, and then, after sending, to play the GUI version of the game. 3 The layout of the experiment and the experiment design used were similar to those reported in the previous section (including the enforcement mechanism for insuring that students will not share their strategy with classmates). In particular, we used the same enforcement system described in Sect.…”

Section: Strategy Descriptionmentioning

confidence: 99%

“…Unlike expert-designed agents, PDAs are developed by nondomain experts, where the goal is to exhibit human-like rather than optimal behavior in a given domain. As such, PDA technology has been suggested in recent years to replace people in system evaluation [2] in various domains such as negotiation [1], costly information gathering [3], security systems [4] and parking allocation [5]. Another common use of PDAs is in studying individual decision making [6].…”

Section: Introductionmentioning

confidence: 99%

“…Nevertheless, despite the great interest in this technology, the applicability of PDA technology was evaluated, to date, mostly through measuring the similarity between the behaviors exhibited by PDAs and their developers, either at the macro level, i.e., comparing the collective or "average" behavior [1,7], or at the micro level, i.e., comparing individual behaviors in similar decision situations [3,5]. No prior research, to the best of our knowledge, has attempted to investigate whether a developer's strategy undergoes some kind of transformation along the process.…”

Section: Introductionmentioning

confidence: 99%

“…This kind of analysis for the outcome performance measure is futile, since this measure, when taken individually, highly depends on chance 3. The idea was to equalize conditions, so that people would have the chance to thoroughly think through their stated strategy, much like how PDA development takes some time.…”

mentioning

confidence: 99%

See 2 more Smart Citations

Agent development as a strategy shaper

Elmalech

Auton Agent Multi-Agent Syst

Agmon

2015

Self Cite

This paper studies to what extent agent development changes one's own strategy. While this question has many general implications it is of special interest to the study of peer designed agents (PDAs), which are computer agents developed by non-experts. This latter emerging technology, has been widely advocated in recent literature for the purpose of replacing people in simulations and investigating human behavior. Its main premise is that strategies programmed into these agents reliably reflect, to some extent, the behavior used by their programmers in real life. We show that PDA development has an important side effect that has not been addressed to date-the process that merely attempts to capture one's strategy is also likely to affect the developer's strategy. This result has many implications concerning the appropriate design of PDA-based simulations as well as the validity of using PDAs for studying individual decision making. The phenomenon is demonstrated experimentally, using two very different application domains and several performance measures. Our findings suggest that the effects on one's strategy arise both in situations where it is potentially possible for people to reason about the optimal strategy (in which case PDA development will enhance the use of an optimal strategy) and in those where calculating the optimal strategy is computationally challenging (in which case PDA development will push people to use more effective strategies, on average). Since in our experiments PDA development actually improved the developer's strategy, PDA development can be suggested as a means for improving people's problem solving skills. Finally, we show that the improvement achieved in people's strategies through agent development is not attributed to the expressive aspect of agent development per-se but rather there is a crucial additional gain in the process of designing and programming ones strategy into an agent.

Problem restructuring for better decision making in recurring decision situations

Elmalech

Auton Agent Multi-Agent Syst

Grosz

2014

Self Cite

This paper proposes the use of restructuring information about choices to improve the performance of computer agents on recurring sequentially dependent decisions. The intended situations of use for the restructuring methods it defines are website platforms such as electronic marketplaces in which agents typically engage in sequentially dependent decisions. With the proposed methods, such platforms can improve agents' experience, thus attracting more customers to their sites. In sequentially-dependent-decisions settings, decisions made at one time may affect decisions made later; hence, the best choice at any point depends not only on the options at that point, but also on future conditions and the decisions made in them. This "problem restructuring" approach was tested on sequential economic search, which is a common type of recurring sequentially dependent decision-making problem that arises in a broad range of areas. The paper introduces four heuristics for restructuring the choices that are available to decision makers in economic search applications. Three of these heuristics are based on characteristics of the choices, not of the decision maker. The fourth heuristic requires information about a decision-makers prior decision-making, which it uses to classify the decision-maker. The classification type is used to choose the best of the three other heuristics. The heuristics were extensively tested on a large number of agents designed by different people with skills similar to those of a typical agent developer. The results demonstrate that the problem-restructuring approach is a promising one for improving the performance of agents on sequentially dependent decisions. Although there was a minor degradation in performance for a small portion of the agents, the overall and average individual performance improved substantially. Complementary experimentation with people demonstrated that the methods carry over, to some extent, also to human decision makers. Interestingly, the heuristic that adapts based on a decision-maker's history achieved the best results for computer agents, but not for people.