Behavioral Modeling Based on Probabilistic Finite Automata: An Empirical Study

Abstract

Imagine an agent that performs tasks according to different strategies. The goal of Behavioral Recognition (BR) is to identify which of the available strategies is the one being used by the agent, by simply observing the agent’s actions and the environmental conditions during a certain period of time. The goal of Behavioral Cloning (BC) is more ambitious. In this last case, the learner must be able to build a model of the behavior of the agent. In both settings, the only assumption is that the learner has access to a training set that contains instances of observed behavioral traces for each available strategy. This paper studies a machine learning approach based on Probabilistic Finite Automata (PFAs), capable of achieving both the recognition and cloning tasks. We evaluate the performance of PFAs in the context of a simulated learning environment (in this case, a virtual Roomba vacuum cleaner robot), and compare it with a collection of other machine learning approaches.

Learning Behavior Trees for Autonomous Agents with Hybrid Constraints Evolution

“…On the other side of the spectrum, various approaches are emerged in machine learning community to generate adaptive agent behavior automatically [5,[20][21][22]. This field has been studied basically from two perspectives [1,2]: learning from observation (LfO) (a.k.a, learning from demonstration, programming from demonstration) and learning from experience. The former allows agent to extract the behavior model of the target agent by observing the behavior trace of another agent (e.g., using NN and case-based learning) [4,20].…”

“…Modern training, entertainment and education applications make extensive use of autonomously controlled virtual agents or physical robots [1]. In these applications, the agents must display complex intelligent behaviors to carry out given tasks.…”

“…However, these ways may not only impose intensive work on human designers when facing multiple types of agents, missions or scenarios, but also result in rigid and predictable agent behaviors [2,3]. An alternative way is using machine learning (ML) techniques to generate agent behaviors automatically [1,4,5]. Through providing sample traces or evaluation criterion of experts' desired behavior, an agent can learn behavior model from expert demonstration or trial-and-error experience respectively.…”

User Interaction Modeling and Profile Extraction in Interactive Systems: A Groupware Application Case Study

“…Determining whether two users have a similar behavior can then be done via distance measures or similarity measures. If users are represented as paths of variable length, their similarity can be calculated using a Monte Carlo estimation of the crossed entropy between their respective traces (see Equation (7) in [ 18 ] or Equation (11) in [ 19 ]). where and is the indicator function of the set .…”

“…Moreover, to improve estimations when the available traces are too short and do not cover the entire set of available actions (along with their respective time intervals), we can use Laplace smoothing, and modify the equation as follows (see Equation (8) in [ 18 ] or Equation (12) in [ 19 ]).…”