A goal-oriented interface to consumer electronics using planning and commonsense reasoning

Abstract

We are reaching a crisis with design of user interfaces for consumer electronics. Flashing 12:00 time indicators, push-andhold buttons, and interminable modes and menus are all symptoms of trying to maintain a one-to-one correspondence between functions and physical controls, which becomes hopeless as the number of capabilities of devices grows. We propose instead to orient interfaces around the goals that users have for the use of devices.We present Roadie, a user interface agent that provides intelligent context-sensitive help and assistance for a network of consumer devices. Roadie uses a Commonsense knowledge base to map between user goals and functions of the devices, and an AI partial-order planner to provide mixed-initiative assistance with executing multi-step procedures and debugging help when things go wrong.

Designing a goal-oriented smart-home environment

“…The user employs an external application (which acts as an interface agent) to introduce the user's desires as an active goal into the DGOC framework. An electronic interface that employs the OpenMind Commonsense knowledge base [19] and ConceptNet [20] is used to transform the user's desires and preferences into goals of the framework [21]. There are also other interfaces available that employ natural language processing (like Siri on Apple devices) or gesture recognition (like Kinect on XBOX from Microsoft) that could be used by users to introduce their goals.…”

Introduction to Intelligent Decision Support Systems

“…Finally, for the independent operation of gazing and the BCI, a mode selection method is proposed that measures a user's concentration by analyzing the pupil accommodation speed, which is not affected by the operation of gazing and the BCI. Lieberman and Espinosa (2007) present Roadie, a user interface agent that provides intelligent context-sensitive help and assistance for a network of consumer devices. Using a commonsense database, the system maps user goals and device functionality.…”

“…Thereafter, an artificial intelligence partial order planner provides two forms of aid to the user. It offers mixed-initiative help with executing multistep procedures, and when something goes wrong, assistance in determining the source of the problem (Lieberman and Espinosa 2007).…”

Augmenting kitchen appliances with a shared context using knowledge about daily events

“…KitchenSense seeks to infer what will be useful or productive to the user through a Goal-Oriented Interface [8], so that when a person opens the freezer and goes to the microwave, KitchenSense decides he probably wants to defrost food. The system constantly infers causal and temporal relationship of events by using spreading activation algorithm (see Fig 4.) Events are decomposed into a directed graph of action nodes.…”