A Reputation–Oriented Reinforcement Learning Strategy for Agents in Electronic Marketplaces

Abstract

In this paper, we propose a reputation-oriented reinforcement learning algorithm for buying and selling agents in electronic market environments. We take into account the fact that multiple selling agents may offer the same good with different qualities. In our approach, buying agents learn to avoid the risk of purchasing low-quality goods and to maximize their expected value of goods by dynamically maintaining sets of reputable sellers. Selling agents learn to maximize their expected profits by adjusting product prices and by optionally altering the quality of their goods. Modeling the reputation of sellers allows buying agents to focus on those sellers with whom a certain degree of trust has been established. We also include the ability for buying agents to optionally explore the marketplace in order to discover new reputable sellers. As detailed in the paper, we believe that our proposed strategy leads to improved satisfaction for buyers and sellers, reduced communication load, and robust systems. In addition, we present preliminary experimental results that confirm some potential advantages of the proposed algorithm, and outline planned future experimentation to continue the evaluation of the model.

Graph Patterns, Reinforcement Learning and Models of Reputation for Improving Coalition Formation in Collaborative Multi-agent Systems

“…Decision-making of goal-or utility-driven autonomous agents based on those agents' view of other agents' reputation has been studied in the context of economics-centric applications of MAS and AI such as, for instance, e-Commerce [16]. However, we argue that quantifying and maintaining reputation of other agent is potentially very beneficial for autonomous agents in a much broader variety of MAS applications, including but not limited to the traditional collaborative domains such as team robotics and ensembles of autonomous unmanned vehicles [3,4].…”

A decentralized trustworthiness estimation model for open, multiagent systems (DTMAS)

“…In other words, these agents get some input, take an action, and receive some reward. Indeed, this framework is the same framework used in reinforcement learning, which is why we chose reinforcement learning as a learning method for our proposed agents model [10].…”

“…Q-learning based trustworthiness estimation is deemed suitable for uncertain environments in an electronic marketplace where agents discover which actions yield the most satisfying results via a trial-and-error search [34]; therefore, we make use of Q-learning for trustworthiness estimation in DTMAS, for simplicity, we will use RL to refer to Q-learning in this work.…”

Fast and Robust Incremental Action Prediction for Interactive Agents

“…In (Tran and Cohen 2002), an agent models the reputation of sellers in an electronic marketplace to more optimally engage in trade. This technique, while interesting and useful, does not address the problem of interest in this paper since it only models a high-level aspect of the other agents, rather than their actual behaviors.…”