Demonstrational Interfaces

Myers, Brad A.; McDaniel, Richard G.

doi:10.1016/b978-155860688-3/50004-x

Cited by 6 publications

(2 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, the referee also serves a second purpose. A well-known challenge in demonstration-based systems is that they may require excessive developer time to provide a sufficient number of demonstrations Lau [2009] or that it may be difficult to provide samples that are sufficiently different from each other Myers and McDaniel [2001], Lee et al [2017]. By continuously evaluating the execution of a currently-trained agent and identifying failing tasks, the referee guides users towards collecting new demonstrations only for critical scenarios, thus reducing the number of manual demonstrations.…”

Section: Referee Modelmentioning

confidence: 99%

Thoracoscopic repair of esophageal atresia

Lin

Duh

et al. 2018

Formosan Journal of Surgery

View full text Add to dashboard Cite

An automation system that can execute natural language instructions by driving the user interface (UI) of an application can benefit users, especially when situationally or permanently impaired. Traditional automation systems (manual scripting, programming by demonstration tools, etc.) do not produce generalizable models that can tolerate changes in the UI or task workflow. Machine-learned automation agents generalize better, but either work only in simple, handcrafted applications or rely on large pre-trained models, which may be too computationally expensive to run on mobile devices. In this paper, we propose UINav, a demonstration-based agent maker system. UINav agents are lightweight enough to run on mobile devices, yet they achieve high success rates with a modest number of task demonstrations. To minimize the number of task demonstrations, UINav includes a referee model that allows users to receive immediate feedback on tasks where the agent is failing to best guide efforts to collect additional demonstrations. Further, UINav adopts macro actions to reduce an agent's state space, and augments human demonstrations to increase the diversity of training data. Our evaluation demonstrates that with an average of 10 demonstrations per task UINav can achieve an accuracy of 70% or higher, and that with enough demonstrations it can achieve near-perfect success rates on 40+ different tasks.

show abstract

Section: Referee Modelmentioning

confidence: 99%

Thoracoscopic repair of esophageal atresia

Lin

Duh

et al. 2018

Formosan Journal of Surgery

View full text Add to dashboard Cite

show abstract

“…Interactive machine learning builds on a long tradition of programming by example (e.g. [25,23]) but it also differs in focusing on statistical learning algorithms and also in not attempting to model general programming, but limiting training by example to specific elements such as classifiers that are well modeled by current learning techniques. The term Interactive Machine Learning (IML) was introduced by [6] who saw it as a way of involving users more closely in the machine learning process by interactively supplying and editing training data.…”

Section: Interactive Machine Learningmentioning

confidence: 99%

Embodied design of full bodied interaction with virtual humans

Gillies

Brenton²,

Kleinsmith

2015

Proceedings of the 2nd International Workshop on Movement and Computing

View full text Add to dashboard Cite

This paper presents a system that allows end users to design full body interactions with 3D animated virtual character through a process we call Interactive Performance Capture. This process is embodied in the sense that users design directly by moving and interacting using an interactive machine learning method. Two people improvise an interaction based only on their movements, one plays the part of the virtual character the other plays a real person. Their movements are recorded and they label it with metadata that identifies certain actions and responses. This labelled data is then used to train a Gaussian Mixture Model that is able to recognized new actions and generate suitable responses from the virtual character. A small study showed that users do indeed design in a very embodied way using movement directly as a means of thinking through and designing interactions.

show abstract

Defining behaviours for solids in a visual design environment

Banyasad

Cox

Proceedings IEEE 2002 Symposia on Human Centric Computing Languages and Environments

View full text Add to dashboard Cite

Demonstrational Interfaces

Cited by 6 publications

References 5 publications

Thoracoscopic repair of esophageal atresia

Thoracoscopic repair of esophageal atresia

Embodied design of full bodied interaction with virtual humans

Defining behaviours for solids in a visual design environment

Contact Info

Product

Resources

About