Negotiation as a Metaphor for Distributed Problem Solving

In this paper, we propose a reputation-oriented reinforcement learning algorithm for buying and selling agents in electronic market environments. We take into account the fact that multiple selling agents may offer the same good with different qualities. In our approach, buying agents learn to avoid the risk of purchasing low-quality goods and to maximize their expected value of goods by dynamically maintaining sets of reputable sellers. Selling agents learn to maximize their expected profits by adjusting product prices and by optionally altering the quality of their goods. Modeling the reputation of sellers allows buying agents to focus on those sellers with whom a certain degree of trust has been established. We also include the ability for buying agents to optionally explore the marketplace in order to discover new reputable sellers. As detailed in the paper, we believe that our proposed strategy leads to improved satisfaction for buyers and sellers, reduced communication load, and robust systems. In addition, we present preliminary experimental results that confirm some potential advantages of the proposed algorithm, and outline planned future experimentation to continue the evaluation of the model.

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A Reputation–Oriented Reinforcement Learning Strategy for Agents in Electronic Marketplaces

Tran

Cohen

2002

“…Distributed negotiation is an approach which is well-suited to the modeling of supply chain formation: each individual procurement and sale decision by each participant in the supply chain can be modeled as a multiparty negotiation, with bids or offers allowing participants to express their capabilities and preferences to potential exchange partners. The Contract Net protocol (Davis and Smith 1983), a technique for distributed problem-solving based on task decomposition and negotiation, formed the basis of many agent-based models of distributed negotiation. While the usefulness of the standard Contract Net protocol to supply chain formation is limited by its myopic, greedy approach and subsequent inability to deal with resource contention, a number of other negotiation-based approaches have been applied to the supply chain formation problem.…”

Section: Supply Chain Formationmentioning

confidence: 99%

Decentralized Supply Chain Formation Using Max‐sum Loopy Belief Propagation

Winsper

Chli

2012

Supply chain formation is the process by which a set of producers within a network determine the subset of these producers able to form a chain to supply goods to one or more consumers at the lowest cost. This problem has been tackled in a number of ways, including auctions, negotiations, and argumentation-based approaches. In this paper we show how this problem can be cast as an optimization of a pairwise cost function. Optimizing this class of energy functions is NP-hard (Boykov et al., 2001) but efficient approximations to the global minimum can be obtained using loopy belief propagation (LBP). Here we detail a max-sum LBP-based approach to the supply chain formation problem, involving decentralized message-passing between supply chain participants. Our approach is evaluated against a well-known decentralized double-auction method and an optimal centralized technique, showing several improvements on the auction method: it obtains better solutions for most network instances which allow for competitive equilibrium † while also optimally solving problems where no competitive equilibrium exists, for which the double-auction method frequently produces inefficient solutions.

“…Our market environment is rooted in an information delivery infrastructure such as the Internet, which provides agents with virtually direct and free access to all other agents. The process of buying and selling goods is realized via a contract-net like mechanism (Smith 1980;Davis and Smith 1983) which consists of three elementary phases: (i) when a buyer b is in need of some good g, it will announce its request for that good to all the sellers, using multi-cast or possibly broadcast. (ii) After receiving the request from b, those sellers that have good g available for sales will send a message to b, stating their price bids for delivering the good.…”

Section: The Agent Market Modelmentioning

confidence: 99%

The Advantages of Designing Adaptive Business Agents Using Reputation Modeling Compared to the Approach of Recursive Modeling

Tran

Cohen

2004

Adaptive business agents operate in electronic marketplaces, learning from past experiences to make effective decisions on behalf of their users. How best to design these agents is an open question. In this article, we present an approach for the design of adaptive business agents that uses a combination of reinforcement learning and reputation modeling. In particular, we take into account the fact that multiple selling agents may offer the same good with different qualities, and that selling agents may alter the quality of their goods. We also consider the possibility of dishonest agents in the marketplace. Our buying agents exploit the reputation of selling agents to avoid interaction with the disreputable ones, and therefore to reduce the risk of purchasing low value goods. We then experimentally compare the performance of our agents with those designed using a recursive modeling approach. We are able to show that agents designed according to our algorithms achieve better performance in terms of satisfaction and computational time and as such are well suited for the design of electronic marketplaces.