Staging transformations for multimodal web interaction management

Abstract

Multimodal interfaces are becoming increasingly ubiquitous with the advent of mobile devices, accessibility considerations, and novel software technologies that combine diverse interaction media. In addition to improving access and delivery capabilities, such interfaces enable flexible and personalized dialogs with websites, much like a conversation between humans. In this paper, we present a software framework for multimodal web interaction management that supports mixed-initiative dialogs between users and websites. A mixed-initiative dialog is one where the user and the website take turns changing the flow of interaction. The framework supports the functional specification and realization of such dialogs using staging transformations -a theory for representing and reasoning about dialogs based on partial input. It supports multiple interaction interfaces, and offers sessioning, caching, and co-ordination functions through the use of an interaction manager. Two case studies are presented to illustrate the promise of this approach. Categories and Subject

A Generative Programming Approach to Interactive Information Retrieval:Insights and Experiences

“…The representation is the only component which contains sitespecific information and is supported as plug-in basis (or simply stored on the original website's server). For further implementation details, including caching and sessioning, see [4]. We have applied these techniques and our framework to several websites: (i) GAMS (Guide to Available Mathematical Software) at gams.nist.gov, (ii) Project Vote Smart at vote-smart.org, (iii) CITIDEL (Computing and Information Technology Interactive Digital Educational Library) at citidel.org [21], (iv) the Open Directory Project at dmoz.org, and (v) the Online Virginia Tech Timetables of Classes accessible through vt.edu.…”

“…We have applied these techniques and our framework to several websites: (i) GAMS (Guide to Available Mathematical Software) at gams.nist.gov, (ii) Project Vote Smart at vote-smart.org, (iii) CITIDEL (Computing and Information Technology Interactive Digital Educational Library) at citidel.org [21], (iv) the Open Directory Project at dmoz.org, and (v) the Online Virginia Tech Timetables of Classes accessible through vt.edu. We refer the reader to [3,4,17] for the details of these case studies.…”

“…This led us to investigate other program transformers (e.g., currying) and study their staging properties. We are now able to develop complex (web) dialogs as compositions of these primitive stagers [4], especially those involving mixed-initiative interaction. This paper begins by presenting these notions.…”

“…This notation provided a concise way to specify complex dialogs (i.e., much more compact than enumerating each individual interaction sequence to be supported). In addition, using a small set of reduction rules [4], which indicate how any dialog (described in this notation) should be simplified each time a user supplies an input, we are able to stage a variety of web dialogs in our generative framework.…”

“…4 (left) wrt the soccer variable at all program points. This leads to the (page assignment) statements at lines 4,6,8,13, and 20 from which we backward slice the program. The result is a representation of the site containing only paths involving hyperlinks labeled 'soccer' leading to leaf webpages containing information about soccer (ref.…”