Fast and Robust Incremental Action Prediction for Interactive Agents

Abstract

The ability for a given agent to adapt on‐line to better interact with another agent is a difficult and important problem. This problem becomes even more difficult when the agent to interact with is a human, because humans learn quickly and behave nondeterministically. In this paper, we present a novel method whereby an agent can incrementally learn to predict the actions of another agent (even a human), and thereby can learn to better interact with that agent. We take a case‐based approach, where the behavior of the other agent is learned in the form of state–action pairs. We generalize these cases either through continuous k‐nearest neighbor, or a modified bounded minimax search. Through our case studies, our technique is empirically shown to require little storage, learn very quickly, and be fast and robust in practice. It can accurately predict actions several steps into the future. Our case studies include interactive virtual environments involving mixtures of synthetic agents and humans, with cooperative and/or competitive relationships.

Online action adaptation in interactive computer games

“…Another relevant approach to opponent modeling is incremental action prediction [Dinerstein et al 2005]. It was designed specifically for use in real-time interactive simulations and provides the ability to model an observed entity and predict its future actions.…”

“…It employs an unsupervised online learning approach to action adaptation in interactive simulations (specifically FPS computer games), however it would be applicable to other interactive applications that allow states and actions to be observed and identified (e.g., intelligent tutoring and surveillance systems). The architecture builds upon incremental case-based approaches to modeling an observed entity [Dinerstein et al 2005;Fagan and Cunningham 2003;and Kerkez and Cox [2001]) and predicts using practical case retrieval and generalization methods. During an encounter between an observing agent and an observed entity, the agent records cases of the environment and the entity's actions.…”

“…Dinerstein et al [2005] propose the use of the relative distance between an observed entity and its nearest opponent, the velocities of the observed entity and the nearest opponent, and an angle representing the average direction towards obstacles as relevant state features, to approximate the state space of FPS/action games. We also make use of these features for primary state information.…”

“…In this article we address the problem of online action-learning and adaptation through the use of incremental case-based approaches to modeling and predicting an observed entity [Dinerstein et al 2005;Fagan and Cunningham 2003;and Kerkez and Cox 2001]. In contrast to previous work, our actionprediction architecture uses a dual-state representation (case-based) to enhance case matching.…”

“…This can be achieved in a variety of ways. For example, actions can be generalized using the k-nearest neighbor algorithm [Dinerstein et al 2005]. To predict several time steps into the future, the retrieval and prediction process is repeated.…”