Decoupling Cognitive Agents and Virtual Environments

Abstract

Abstract. The development of and accessibility to rich virtual environments, both for recreation and training activities leads to the use of intelligent agents to control avatars (and other entities) in these environments. There is a fundamental tension in such systems between tight integration, for performance and low coupling, for generality, flexibility and extensibility. This paper addresses the engineering issues in connecting agent platforms and other software entities with virtual environments, driven by the following informal requirements: (i) accessibility: we would like (easily) to be able to connect any (legacy) software component with the virtual environment (ii) performance: we want the benefits of decoupling, but not at a high price in performance (iii) distribution: we would like to be able to locate functionality where needed, when necessary, but also be location agnostic otherwise (iv) scalability: we would like to support large-scale and geographically dispersed virtual environments. We start from the position that the basic currency unit of such systems can be events. We describe the Bath Sensor Framework, which is a middleware that attempts to satisfy the above goals and to provide a low-latency linking mechanism between event producers and event consumers, while minimising the effect of coupling of components. We illustrate the framework in two complementary case studies using the Jason agent platform, Second Life and AGAVE (a 3D VE for vehicles). Through these examples, we are able to carry out a preliminary evaluation of the approach against the factors above, against alternative systems and demonstrate effective distributed execution.

A Situational Awareness Approach to Intelligent Vehicle Agents

“…These informal requirements have lead to the creation of a distributed environment called the Bath Sensor Framework (BSF) [11], whose primary features are: (i) a bus-like communications system based on the eXtended Messaging and Presence Protocol (XMPP) [19], and (ii) a publish/subscribe interface that can be implemented for a variety of programming languages (we currently use Java, C# and Python). Similar XMPP-based approaches have been demonstrated in other distributed applications [20,21].…”

Blur robust optical flow using motion channel

“…Furthermore, our tracker is proposed to provide the rotation (yaw, pitch and roll), translation and zoom information within a reasonable error range (2 mm). To synchronise this tracker data and the image recording, a real time collaboration (RTC) server [22] is built using the instant messaging protocol XMPP (also known as Jabber 1 ) which is designed for messageoriented communication based on XML, and allows real-time responses between different messaging channels or any signal channels that can be transmitted and received in message form. In this case, a time stamp is assigned to the received message package by the central timer of the server.…”

Robust optical flow estimation for continuous blurred scenes using RGB-motion imaging and directional filtering

“…2(a)), combining an RGB sensor and a 3D pose&position tracker (Trackir, NaturalPoint Inc.) in order to capture continuous scenes along with real-time camera pose&position information. Our tracker provides the rotation (yaw, pitch and roll), translation and zoom information synchronized to the relative corresponding image frame using the middleware of [8]. Assuming objects have similar depth within the same scene (A common assumption in image deblurring which will be discussed in our future work), the tracked 3D camera motion in image coordinates can be formulated as:…”