Enhancing Mobile Voice Assistants with WorldGaze

Mayer, Sven; Laput, Gierad; Harrison, Chris

doi:10.1145/3313831.3376479

Cited by 39 publications

(30 citation statements)

References 47 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In industry, several different smartphone applications have been released that incorporate face tracking via the frontfacing camera for experiences like Animoji, Memoji, and face filters [3,14,48]. Recent works have adopted RGB [1,8,17] or depth camera [5,34] to accurately track the user's head pose or gaze. These methods apply vision-based gaze or head tracking, which carry privacy concerns [22], and require the user to be in the field of view of the front-facing camera.…”

Section: Attentive User Interfacesmentioning

confidence: 99%

“…In particular, users tend to orient their heads toward their intended interaction targets [49]. To recognize user interaction intention, researchers have leveraged eye-gaze or head tracking by using dedicated devices [12,24] or by using an external camera [8,10,34]. These methods apply vision-based gaze or head tracking, which carry privacy concerns [22], and require the user to be in the field of view of the front-facing camera.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

FaceOri: Tracking Head Position and Orientation Using Ultrasonic Ranging on Earphones

Wang,

Ding,

Chatterjee

et al. 2022

Preprint

View full text Add to dashboard Cite

Face orientation can often indicate users' intended interaction target. In this paper, we propose FaceOri, a novel face tracking technique based on acoustic ranging using earphones. FaceOri can leverage the speaker on a commodity device to emit an ultrasonic chirp, which is picked up by the set of microphones on the user's earphone, and then processed to calculate the distance from each microphone to the device. These measurements are used to derive the user's face orientation and distance with respect to the device. We conduct a ground truth comparison and user study to evaluate FaceOri's performance. The results show that the system can determine whether the user orients to the device at a 93.5% accuracy within a 1.5 meters range. Furthermore, FaceOri can continuously track user's head orientation with a median absolute error of 10.9 mm in the distance, 3.7 • in yaw, and 5.8 • in pitch. FaceOri can

show abstract

Section: Attentive User Interfacesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

FaceOri: Tracking Head Position and Orientation Using Ultrasonic Ranging on Earphones

Wang,

Ding,

Chatterjee

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Xiao et al [ 122 ] attempted to develop a prototype of an active-listening chatbot to interview people. WorldGaze [ 123 ] tries to leverage the existing cameras of smartphones to include gaze information to enrich voice assistants. Jensen et al [ 124 ] present a system to generate feedback for teachers in classrooms automatically.…”

Section: Classifying Hcml Researchmentioning

confidence: 99%

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

Kaluarachchi

Reis

Nanayakkara

2021

Sensors

View full text Add to dashboard Cite

After Deep Learning (DL) regained popularity recently, the Artificial Intelligence (AI) or Machine Learning (ML) field is undergoing rapid growth concerning research and real-world application development. Deep Learning has generated complexities in algorithms, and researchers and users have raised concerns regarding the usability and adoptability of Deep Learning systems. These concerns, coupled with the increasing human-AI interactions, have created the emerging field that is Human-Centered Machine Learning (HCML). We present this review paper as an overview and analysis of existing work in HCML related to DL. Firstly, we collaborated with field domain experts to develop a working definition for HCML. Secondly, through a systematic literature review, we analyze and classify 162 publications that fall within HCML. Our classification is based on aspects including contribution type, application area, and focused human categories. Finally, we analyze the topology of the HCML landscape by identifying research gaps, highlighting conflicting interpretations, addressing current challenges, and presenting future HCML research opportunities.

show abstract

“…For example, the augmented sensory capabilities from epidermal and on-body interfaces can augment the conversation design by knowing the environmental context in the surroundings of the users. In WorldGaze [191], the conversational agent leverages the smartphone camera to enhance its context awareness, where the enhanced knowledge of the user's situation makes more intuitive user interaction with the conversational interfaces. As shown in Table 2 (Appendix), the design space of CUIs should consider the wider aspects of networking and AR interfaces.…”

Section: Conversational User Interfacementioning

confidence: 99%

Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges

et al. 2021

View full text Add to dashboard Cite

Interaction design for Augmented Reality (AR) is gaining attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015–2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, followed by reviewing the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and input techniques that warrant further research and conjecture that AR with complementary Conversational User Interfaces (CUIs) is a crucial enabler for ubiquitous interaction with immersive systems in smart cities. Our work helps researchers understand the current potential and future needs of AR in Human-City Interaction.

show abstract

Enhancing Mobile Voice Assistants with WorldGaze

Cited by 39 publications

References 47 publications

FaceOri: Tracking Head Position and Orientation Using Ultrasonic Ranging on Earphones

FaceOri: Tracking Head Position and Orientation Using Ultrasonic Ranging on Earphones

A Review of Recent Deep Learning Approaches in Human-Centered Machine Learning

Towards Augmented Reality Driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges

Contact Info

Product

Resources

About