Voice enabling mobile applications with UIVoice

Tarakji, Ahmad Bisher; Xu, Jing; Colmenares, Juan A.; Mohomed, Iqbal

doi:10.1145/3213344.3213353

Cited by 8 publications

(6 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Speech difficulties are one of the key factors that affect an individual with DS's ability to communicate with others and eventually impact the free movement in local settings or around a city [15,36]. The advancement of voice assistance revealed the possibility of using such technology to support people with cognitive disabilities [31,70,77]. Carroll et al [18] designed a speech-based prototype of a context-aware assistive application called Robin on Amazon's Alexa platform to support routing activities by prompt audio for people with dementia.…”

Section: Outdoor Safety and Voice Assistantsmentioning

confidence: 99%

“…Consequently, voice assistants can be eventually enabled in the assistive app. Tarakji et al [70] proposed a prototype of the framework to create Voice User Interfaces (VUIs) on top of existing applications to enable voice assistance with mobile applications such as YouTube, WhatsApp and LinkedIn. Many studies have been carried out that involved people with cognitive disability to evaluate how to increase their independence.…”

Section: Outdoor Safety and Voice Assistantsmentioning

confidence: 99%

“…Balasuriya et al, Carroll et al, Chang et al and Tarakji et al [9,18,19,70]. Likewise, the need for staying in touch with caregivers via phone calls were described by Carroll et alDawe et al, Lazar et al and Lazar et al [18,25,50,51].…”

mentioning

confidence: 99%

See 2 more Smart Citations

Towards Designing Mobile Apps for Independent Travel

Khan

Dunlop

McGee‐Lennon

et al. 2021

ACM Trans. Access. Comput.

View full text Add to dashboard Cite

Performing daily independent activities can be a challenge for people with Down's Syndrome (DS). This article investigates how to better support these activities with smart devices based on three cycles of a collaborative participatory action research (PAR) process. The first cycle involved semi-structured interviews ( n = 4 ) with parents and an online survey ( n = 39 ) with people with DS and their parents to explore barriers and opportunities for independent activities. This cycle highlighted that travelling independently was a common challenge among discussed barriers to independent activities for young adults with DS, an issue that smart devices have the potential to overcome. The second cycle involved seven focus group discussions ( n = 20 ) with parents ( n = 13 ) and young adults with DS ( n = 7 ) for gaining deeper insights into barriers to independent travel. We explored key barriers to independent travel and gathered design requirements for smartphone apps to overcome these barriers. In the third cycle, we designed a digital prototype based on participant recommendations and conducted seven focus group meetings ( n = 19 ) with caregivers ( n = 12 ) and individuals with DS ( n = 7 ). This final cycle reviewed the proposed digital prototype and validated the key barriers found in the second cycle. Overall, our studies confirmed that mobile technology can support people with DS in performing daily life activities that increase social inclusion. The studies resulted in identified barriers and requirements along with co-designed solutions for independent travel apps.

show abstract

Section: Outdoor Safety and Voice Assistantsmentioning

confidence: 99%

Section: Outdoor Safety and Voice Assistantsmentioning

confidence: 99%

See 1 more Smart Citation

Towards Designing Mobile Apps for Independent Travel

Khan

Dunlop

McGee‐Lennon

et al. 2021

ACM Trans. Access. Comput.

View full text Add to dashboard Cite

show abstract

“…Among the most recent proponents of this idea in web browsing, Baez, Daniel and Casati 7 propose full conversational access to websites where a chatbot mediates the interaction between the user and the website, allowing users to express their goals in natural language; while Ripa et al 8 focus on making informational queries over content intensive websites accessible via voice-based interfaces (e.g., smart speakers). For mobile applications, Tarakji et al 9 propose a framework that allows third-party developers to create voice user interfaces for existing Android apps, thus enabling users to interact with the apps in their mobile phones from smart speakers.…”

Section: Conversational Gui Agentmentioning

confidence: 99%

Chatbot Integration in Few Patterns

Báez

Daniel

Casati

et al. 2021

IEEE Internet Comput.

View full text Add to dashboard Cite

Chatbots are software agents that are able to interact with humans in natural language. Their intuitive interaction paradigm is expected to significantly reshape the software landscape of tomorrow, while already today chatbots are invading a multitude of scenarios and contexts. This article takes a developer's perspective, identifies a set of architectural patterns that capture different chatbot integration scenarios, and reviews state-ofthe-art development aids. This paper was written with the precious contribution of co-author Florian Daniel, who passed away a few days after completing this manuscript for submission. We remember him here. 1

show abstract

“…SOVITE bridges an important gap in these systems, as they focus on the ambiguities, uncer tainties, and vagueness embedded in the user instructions of In terms of the technique used, SOVITE extracts the semantics of app GUIs [13,43] for grounding natural language conver sations. Compared with the previous systems that used the semantics of app GUIs for learning new tasks [36,38,58], extracting task flows [40], and supporting invoking individual GUI widgets with voice commands [64], a new idea in SO VITE is that it encodes app GUIs into the same vector space as natural language utterances, allowing the system to look up semantically relevant task intents when the user refers to apps and app GUI screens in the dialogues for repairing intent detection errors (details in the Implementation section).…”

Section: Multi-modal Mixed-initiative Disambiguation Interfacesmentioning

confidence: 99%

Multi-Modal Repairs of Conversational Breakdowns in Task-Oriented Dialogs

Chen

Xia

et al. 2020

Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

A major problem in task-oriented conversational agents is the lack of support for the repair of conversational breakdowns. Prior studies have shown that current repair strategies for these kinds of errors are often ineffective due to: (1) the lack of transparency about the state of the system's understanding of the user's utterance; and (2) the system's limited capabilities to understand the user's verbal attempts to repair natural language understanding errors. This paper introduces SOVITE, a new multi-modal (speech plus direct manipulation) interface that helps users discover, identify the causes of, and recover from conversational breakdowns using the resources of existing mobile app GUIs for grounding. SOVITE displays the system's understanding of user intents using GUI screenshots, allows users to refer to third-party apps and their GUI screens in conversations as inputs for intent disambiguation, and enables users to repair breakdowns using direct manipulation on these screenshots. The results from a remote user study with 10 users using SOVITE in 7 scenarios suggested that SOVITE's approach is usable and effective.

show abstract

Voice enabling mobile applications with UIVoice

Cited by 8 publications

References 6 publications

Towards Designing Mobile Apps for Independent Travel

Towards Designing Mobile Apps for Independent Travel

Chatbot Integration in Few Patterns

Multi-Modal Repairs of Conversational Breakdowns in Task-Oriented Dialogs

Contact Info

Product

Resources

About