Adopting Conversational Interfaces for Exploring OSGi-Based Software Architectures in Augmented Reality

Abstract

We propose conversational interfaces as a convenient and complementary way for users to explore OSGi-based software architectures in immersive Augmented Reality (AR). By providing a conversational interface we aim to remedy some peculiarities of AR devices, but also enhancing the exploration task at hand. We exemplify a use case and sketch how different user utterances can be used to retrieve information about the to-be-explored OSGi-based software architecture. We identify crucial components such as natural language generation and intent recognition which are required to implement the user story and we outline the status of our implementation.

Bots in software engineering: a systematic mapping study

“…This opens up many opportunities in Human–computer Interaction (HCI). IslandViz ( Misiak et al, 2018 ; Seipel et al, 2019 ) lets developers visualize complex software architectures in 3D via Augmented Reality (AR). Speech-based commands are used to navigate and explore specific parts of the visualization in Microsoft Hololens.…”

Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges

“…Voice-based commands enable users to naturally interact with the digital contents in AR, for instance, choosing and manipulating 3D graphics with voice commands [114], indicating the longitudinal (forward and backward), vertical (upward and downward), lateral (right and left) movements for flying drones [153], and making appointment in calendar [131]. Although the voice dialogues between the human user and the system pose issues of ineffective information display, hard to review and edit [138] and social acceptance [52], the voice commands are particularly useful for interaction with AR objects and other smart objects like UAV, as users can obtain instant feedback from the target objects.…”

“…However, we identify improvement areas for more natural CUIs. First, the conversational agent's response conversation cannot be interrupted even though the user is not getting a desired answer in the conversation [114]. Second, the agents only respond to the current questions from a user, which means the prior questions cannot be considered as a continual conversation [108].…”

Unleashing the Potentials of Immersive Augmented Reality for Software Engineering

“…Although little research has focused on investigating the potentials of using IAR to support software engineering, we consider these initial works to be important contributions: For instance, IAR has been employed to support the analysis of software architectures based on visualization [3] and conversational interfaces [4]. IAR has also been used to navigate 3D visualizations to support software comprehension [5] and software performance [6] Figure 1: A software engineer wears a Microsoft HoloLens to extend the space of his laptops' screens by superimposing computer-generated images in the physical environment.…”