Automatic Generation of Chatbots for Conversational Web Browsing

Abstract

In this paper, we describe the foundations for generating a chatbot out of a website equipped with simple, bot-specific HTML annotations. The approach is part of what we call conversational web browsing, i.e., a dialog-based, natural language interaction with websites. The goal is to enable users to use content and functionality accessible through rendered UIs by "talking to websites" instead of by operating the graphical UI using keyboard and mouse. The chatbot mediates between the user and the website, operates its graphical UI on behalf of the user, and informs the user about the state of interaction. We describe the conceptual vocabulary and annotation format, the supporting conversational middleware and techniques, and the implementation of a demo able to deliver conversational web browsing experiences through Amazon Alexa.

Bringing Cognitive Augmentation to Web Browsing Accessibility

“…In our previous work we highlighted the need for supporting application-specific intents, i.e., intents that are specific to the offerings of a website (e.g., "Post a new comment on the news article") and that would trigger a series of lowlevel actions as a result. In our approach such experience required bot-specific annotations [14]. The automation of such workflows has also been explored in the context of Web accessibility.…”

“…There are many challenges in delivering the type of support required for conversational browsing. As discussed in our prior work [14], this requires deriving two important types of knowledge:…”

“…Instead of relying on the sequential navigation and keyboard shortcuts provided by screen readers, this approach would enable BVIP to express their goals by directly "talking to websites". The first step towards this vision was to identify the conceptual vocabulary for augmenting websites with conversational capabilities and explore techniques for generating chatbots out of websites equipped with bot-specific annotations [14].…”