Looks Can Be Deceiving

Newn, Joshua; Allison, Fraser; Velloso, Eduardo; Vetere, Frank

doi:10.1145/3173574.3173835

Cited by 29 publications

(15 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Communication, both verbal and nonverbal, is crucial in social decision making (van Dijk & De Dreu, 2021). We see this in work like that of Newn et al (2018), who employed an online game (Ticket to Ride) and recorded gaze behavior. When an opponent's gaze was displayed, players self-reported actively using it to predict their opponent's intention.…”

Section: Games As a Happy Medium: Ecologically Valid Task With Experi...mentioning

confidence: 98%

“…As such, more investigations are conducted in enriched environments and/or manipulate facets of the physical and social context to more closely mirror the physical and social complexity of the real world (i.e., Foulsham et al, 2011; Hayward et al, 2017). For example, complex social dynamics can be revealed by studying participant pairs (dyads; e.g., Brennan et al, 2008; Forder & Dyson, 2016; Laakasuo et al, 2015; Newn et al, 2018). In these cases, the effects of social context emerge, such as in partnered visual search which yields faster search RTs when sharing gaze than acting alone (Brennan et al, 2008).…”

Section: Classic Work On Uncertainty and Decision Makingmentioning

confidence: 99%

See 1 more Smart Citation

You read my mind: Generating and minimizing intention uncertainty under different social contexts in a two-player online game.

Bertrand

Chapman

et al. 2023

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

We investigate an ecologically pertinent form of social uncertainty regarding the ability to read another’s intentions. We use classic measures (response time, accuracy) and dynamic measures (mouse trajectories) to investigate how people generate or minimize uncertainty regarding their own intentions under different social contexts, and how uncertainty regarding other’s intentions affects decision making. Ninety-six participants (N = 48 dyads) completed a two-player online card game, where the goal was to collect cards with a certain feature (e.g., triangles), with participant cursor movements projected to both players. Participants played six games, three cooperatively and three competitively (Social Decision Context). Points were awarded for two decisions: collecting a card matching one’s goal (ability to achieve personal goal) and correctly guessing the other player’s goal (ability to guess intention). Data revealed: (a) Card scores did not vary with Social Decision Context, (b) Guess scores did vary with Social Decision Context, with more correct guesses when cooperating compared to competing, and (c) Mouse trajectories (durations and mouse distance traveled) decreased when cooperating compared to competing. These results indicate that better guessing during cooperative play is not due to explicit communication (i.e., circling desired cards), but may be due to increased speed and confidence when making decisions in a cooperative context. Additionally, participants could be actively hiding their intention in a competitive context. Thus, social uncertainty when reading another’s intentions is both adaptive—affected by the prescribed social context, and automatic—indirectly inferred from the way another moves their mouse when acting with intention.

show abstract

Section: Games As a Happy Medium: Ecologically Valid Task With Experi...mentioning

confidence: 98%

Section: Classic Work On Uncertainty and Decision Makingmentioning

confidence: 99%

You read my mind: Generating and minimizing intention uncertainty under different social contexts in a two-player online game.

Bertrand

Chapman

et al. 2023

Journal of Experimental Psychology: Human Perception and Perfor

View full text Add to dashboard Cite

show abstract

“…The ability to tell what someone is looking at-'gaze awareness'-is a useful way to gauge the attention of others [1,2,14,62]. Gaze observed over time has been shown to be an effective predictor of human intention [26,27,49,55]. A common approach for gaze awareness is to visually overlay a user's gaze over a shared interface, which provides others rich insights into the mind of the tracked user.…”

Section: Introductionmentioning

confidence: 99%

“…A common approach for gaze awareness is to visually overlay a user's gaze over a shared interface, which provides others rich insights into the mind of the tracked user. This complementary layer of communication has numerous benefits such as improved coordination [2,12,14] and situation awareness [49]. Despite these benefits, overlaying gaze on the interface can add a highly distracting element to the task at hand [49], confuse users when there is a mismatch with other modes of communication such as speech [14], and scales poorly with multiple users.…”

Section: Introductionmentioning

confidence: 99%

“…The agent assists by making inferences about the opponent's intentions based on their gaze and actions, allowing the user to focus on formulating better strategies with improved awareness of the situation. By abstracting the gaze data into a written prediction of what the opponent intends to do, we avoid the distracting nature of gaze visualisation as found in past research [14,49,50,62]. As nonverbal cues are difficult to articulate, our first step was to build a linguistic model of intention recognition derived from human observers.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Designing Interactions with Intention-Aware Gaze-Enabled Artificial Agents

Newn

Singh

Allison

et al. 2019

Human-Computer Interaction – INTERACT 2019

Self Cite

View full text Add to dashboard Cite

As it becomes more common for humans to work alongside artificial agents on everyday tasks, it is increasingly important to design artificial agents that can understand and interact with their human counterparts naturally. We posit that an effective way to do this is to harness nonverbal cues used in humanhuman interaction. We, therefore, leverage knowledge from existing work on gazebased intention recognition, where the awareness of gaze can provide insights into the future actions of an observed human subject. In this paper, we design and evaluate the use of a proactive intention-aware gaze-enabled artificial agent that assists a human player engaged in an online strategy game. The agent assists by recognising and communicating the intentions of a human opponent in real-time, potentially improving situation awareness. Our first study identifies the language requirements for the artificial agent to communicate the opponent's intentions to the assisted player, using an inverted Wizard of Oz method approach. Our second study compares the experience of playing an online strategy game with and without the assistance of the agent. Specifically, we conducted a within-subjects study with 30 participants to compare their experience of playing with (1) detailed AI predictions, (2) abstract AI predictions, and (3) no AI predictions but with a live visualisation of their opponent's gaze. Our results show that the agent can facilitate awareness of another user's intentions without adding visual distraction to the interface; however, the cognitive workload was similar across all three conditions, suggesting that the manner in which the agent communicates its predictions requires further exploration. Overall, our work contributes to the understanding of how to support human-agent teams in a dynamic collaboration scenario. We provide a positive account of humans interacting with an intention-aware artificial agent afforded by gaze input, which presents immediate opportunities for improving interactions between humans and agents and demonstrates the usefulness of human elicitation for producing communication protocols for effective interaction.

show abstract