The Effects of Experience on Deception in Human-Agent Negotiation

Abstract

Negotiation is the complex social process by which multiple parties come to mutual agreement over a series of issues. As such, it has proven to be a key challenge problem for designing adequately social AIs that can effectively navigate this space. Artificial AI agents that are capable of negotiating must be capable of realizing policies and strategies that govern offer acceptances, offer generation, preference elicitation, and more. But the next generation of agents must also adapt to reflect their users’ experiences.
 The best human negotiators tend to have honed their craft through hours of practice and experience. But, not all negotiators agree on which strategic tactics to use, and endorsement of deceptive tactics in particular is a controversial topic for many negotiators. We examine the ways in which deceptive tactics are used and endorsed in non-repeated human negotiation and show that prior experience plays a key role in governing what tactics are seen as acceptable or useful in negotiation. Previous work has indicated that people that negotiate through artificial agent representatives may be more inclined to fairness than those people that negotiate directly. We present a series of three user studies that challenge this initial assumption and expand on this picture by examining the role of past experience.
 This work constructs a new scale for measuring endorsement of manipulative negotiation tactics and introduces its use to artificial intelligence research. It continues by presenting the results of a series of three studies that examine how negotiating experience can change what negotiation tactics and strategies human endorse. Study #1 looks at human endorsement of deceptive techniques based on prior negotiating experience as well as representative effects. Study #2 further characterizes the negativity of prior experience in relation to endorsement of deceptive techniques. Finally, in Study #3, we show that the lessons learned from the empirical observations in Study #1 and #2 can in fact be induced—by designing agents that provide a specific type of negative experience, human endorsement of deception can be predictably manipulated.

Bad machines corrupt good morals

“…People also delegate a growing number of tasks to AI agents 5,104,105 , as diverse as setting prices in online markets 86 , interrogating suspects 106 , or devise a sales strategy 107 . New forms of ethical risks emerge because the delegation of ethically questionable behavior to AI agents might be particularly attractive 108 : The often-incomprehensible workings of algorithms create ambiguity 109,110 . Letting such "black box" algorithms execute tasks on one's behalf increases plausible deniability 10,106 , and obfuscates the attribution of responsibility for the harm caused 111 .…”

Social Functions of Machine Emotional Expressions

“…Interestingly, people concluded that this machine was more likely to cooperate in the future than another which, despite behaving just as selfishly, expressed no emotion. Moreover, whereas some may be quick to condemn deception in machines, a study revealed that some participants endorsed deceptive behavior in machines acting on their behalf during a negotiation task, especially following poor outcomes in prior rounds [180]. In reality, conflict between individual and collective interests permeates human life [181], [182] and it is, in that sense, unsurprising that similar dilemmas would emerge when engaging with machines that may often be representing other humans' conflicting interests.…”

A Survey of Decision Support Mechanisms for Negotiation

“…Two example challenges are the following. Can we detect deception [24,48]? How can agents create rapport with people [12,53,67]?…”