Putting the Con in Context: Identifying Deceptive Actors in the Game of Mafia

Abstract

While neural networks demonstrate a remarkable ability to model linguistic content, capturing contextual information related to a speaker's conversational role is an open area of research. In this work, we analyze the effect of speaker role on language use through the game of Mafia, in which participants are assigned either an honest or a deceptive role. In addition to building a framework to collect a dataset of Mafia game records, we demonstrate that there are differences in the language produced by players with different roles. We confirm that classification models are able to rank deceptive players as more suspicious than honest ones based only on their use of language. Furthermore, we show that training models on two auxiliary tasks outperforms a standard BERTbased text classification approach. We also present methods for using our trained models to identify features that distinguish between player roles, which could be used to assist players during the Mafia game.

Long-Horizon Dialogue Understanding for Role Identification in the Game of Avalon with Large Language Models

“…Unlike prior works which explore deception through analysis of verbal (Hirschberg et al, 2005) or visual (Soldner et al, 2019) cues in spoken language from two-party dialogues, our benchmark is based on textual linguistic cues in multi-party dialogues. Although datasets have previously been introduced for the games of Mafia (Ibraheem et al, 2022) and One Night Werewolf (Lai et al, 2022), we find Avalon to be a significantly more challenging task due to the increased game length, resulting in more than double the number of utterances per game in our dataset -49, 64, and 119 for Mafia, Werewolf, and Avalon, respectively. This requires dialogue models to reason over significantly longer context horizons, but also provides enough information for us to reason over hidden player roles as opposed to simply inferring utterance labels.…”