End-User Development of Voice User Interfaces Based on Web Content

Abstract

Voice Assistants, and particularly the latest gadgets called smart speakers, allow end users to interact with applications by means of voice commands. As usual, end users are able to install applications (also called skills) that are available in repositories and fulfill multiple purposes. In this work we present an end-user environment to define skills for voice assistants based on the extraction of Web content and their organization into different voice navigation patterns. We describe the approach, the end-user development environment, and finally we present some case studies based on Alexa and Amazon Echo.

Chatbot Integration in Few Patterns

“…Approaches to Conversational UI agents can be found notably for Web browsing and mobile applications. Among the most recent proponents of this idea in web browsing, Baez, Daniel and Casati 7 propose full conversational access to websites where a chatbot mediates the interaction between the user and the website, allowing users to express their goals in natural language; while Ripa et al 8 focus on making informational queries over content intensive websites accessible via voice-based interfaces (e.g., smart speakers). For mobile applications, Tarakji et al 9 propose a framework that allows third-party developers to create voice user interfaces for existing Android apps, thus enabling users to interact with the apps in their mobile phones from smart speakers.…”

Open Data Accessibility Based on Voice Commands

“…This approach is specifically committed to help only blind people navigate to contents. In another work [21], the authors presented an approach to specifically allow end users to extract Web contents using voice assistants. However, the access to datasets is not possible because these approaches cannot access external resources such as the tabular data from open data portals.…”

“…Moreover, for visuallyimpaired users who have recently acquired this condition or who have not yet acquired effective browsing strategies, it can be difficult to learn how to access Web contents using screen readers [4]. On the other hand, there are approaches that suggest voice interfaces [2,21] to improve Web accessibility, but these are not easily applicable in open data portals because the requirement to access data from external files (distributions) is not currently considered.…”

Automatic Generation of Chatbots for Conversational Web Browsing

“…Popular in e-commerce and CRM, approaches such as SuperAgent [7] can generate conversational FAQ based on the content to visitors directly on the website. Ripa et al [12] focus on making informational queries over content intensive websites accessible via voice-based interfaces (e.g., smart speakers), relying on augmentations provided by end-users. While all these works illustrate the diversity of approaches, they require either (bot) programming knowledge (and effort), are constrained by an application domain, or are limited to Q&A.…”

“…Integrating conversational capabilities into software enabled services is an emerging research topic [3], as pushed by recent works by Castaldo et al [5] on inferring bots directly from database schemas, Yaghoub-Zadeh-Fard et al [13] on deriving bots from APIs, and by Ripa et al [12] on generating informational bots out of website content. While these works are facilitating chatbot integration at different levels of the Web architecture, they do not address the challenges of generating chatbots from both content and functionality available in websites.…”