Integrating gesture and snapping into a user interface toolkit

Henry, Tyson R.; Hudson, Scott E.; Newell, Gary L.

doi:10.1145/97924.97938

Cited by 53 publications

(21 citation statements)

References 19 publications

(12 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In the same years, Henry et al [56] used FSMs for solving the problem of modelling non atomic actions on the UI, such as the drag and drop technique. Indeed, such kind of interaction is particularly tedious for developers, since they need to track the event sequence in order to implement describe the temporal relationship of the user's actions, which is close to the definition of a gesture.…”

Section: 2mentioning

confidence: 99%

A model-based approach for gesture interfaces

Spano

2011

Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems

View full text Add to dashboard Cite

This one goes out to the one I love "People shouldn't have to read a manual to open a door, even if it is only one word long (push/pull AbstractThe description of a gesture requires temporal analysis of values generated by input sensors, and it does not fit well the observer pattern traditionally used by frameworks to handle the user's input. The current solution is to embed particular gesture-based interactions into frameworks by notifying when a gesture is detected completely. This approach suffers from a lack of flexibility, unless the programmer performs explicit temporal analysis of raw sensors data. This thesis proposes a compositional, declarative meta-model for gestures definition based on Petri Nets. Basic traits are used as building blocks for defining gestures; each one notifies the change of a feature value. A complex gesture is defined by the composition of other sub-gestures using a set of operators. The user interface behaviour can be associated to the recognition of the whole gesture or to any other sub-component, addressing the problem of granularity for the notification of events.The meta-model can be instantiated for different gesture recognition supports and its definition has been validated through a proof of concept library. Sample applications have been developed for supporting multi-touch gestures in iOS and full body gestures with Microsoft Kinect.In addition to the solution for the event granularity problem, this thesis discusses how to separate the definition of the gesture from the user interface behaviour using the proposed compositional approach.The gesture description meta-model has been integrated into MARIA, a model-based user interface description language, extending it with the description of full-body gesture interfaces.

show abstract

Section: 2mentioning

confidence: 99%

A model-based approach for gesture interfaces

Spano

2011

Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems

View full text Add to dashboard Cite

show abstract

“…This functionality allows for "uncluttering" of the display, removing GUI widgets and preserving space for the key focal point of any image review workstation-the images. Gesture recognition itself has been explored in other application contexts, including: handwriting recognition, personal device assistants (PDAs), web browser manipulation, computer drawing programs, and computer games [40]- [44]. The set of gestures is configurable such that a gesture from a training set can be dynamically assigned to one of a set of common image manipulation functions (e.g., window-level setting, rotation, image layout settings, measurement, annotation), allowing the user to tailor the display and the functionality of the application to his/her preferences.…”

Section: ) Application Programmer's Interfacementioning

confidence: 99%

openSourcePACS: An Extensible Infrastructure for Medical Image Management

Bui

Morioka

Dionisio

et al. 2007

IEEE Trans. Inform. Technol. Biomed.

View full text Add to dashboard Cite

Abstract-The development of comprehensive picture archive and communication systems (PACS) has mainly been limited to proprietary developments by vendors, though a number of freely available software projects have addressed specific image management tasks. The openSourcePACS project aims to provide an open source, common foundation upon which not only can a basic PACS be readily implemented, but to also support the evolution of new PACS functionality through the development of novel imaging applications and services. openSourcePACS consists of four main software modules: 1) image order entry, which enables the ordering and tracking of structured image requisitions; 2) an agent-based image server framework that coordinates distributed image services including routing, image processing, and querying beyond the present digital image and communications in medicine (DICOM) capabilities; 3) an image viewer, supporting standard display and image manipulation tools, DICOM presentation states, and structured reporting; and 4) reporting and result dissemination, supplying web-based widgets for creating integrated reports. All components are implemented using Java to encourage cross-platform deployment. To demonstrate the usage of openSourcePACS, a preliminary application supporting primary care/specialist communication was developed and is described herein. Ultimately, the goal of openSourcePACS is to promote the wide-scale development and usage of PACS and imaging applications within academic and research communities.Index Terms-Biomedical imaging, image communication, radiology information system (RIS)/picture archiving and communication system (PACS) fusion, software libraries.

show abstract

“…They also describe input and event handling in a cleaner, more general and extensible way than traditional toolkits [26,15]. Even though they support at some level Post-WIMP techniques such as simple gesture recognition, they are only aware of a limited set of input devices and require significant modifications to handle any new interaction paradigm.…”

Section: Advanced Gui Toolkitsmentioning

confidence: 99%

The MaggLite post-WIMP toolkit

Huot¹,

Dumas²,

Dragicevic

et al. 2004

Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology

View full text Add to dashboard Cite

This article presents MaggLite, a toolkit and sketch-based interface builder allowing fast and interactive design of post-WIMP user interfaces. MaggLite improves design of advanced UIs thanks to its novel mixed-graph architecture that dynamically combines scene-graphs with interactiongraphs. Scene-graphs provide mechanisms to describe and produce rich graphical effects, whereas interaction-graphs allow expressive and fine-grained description of advanced interaction techniques and behaviors such as multiple pointers management, toolglasses, bimanual interaction, gesture, and speech recognition. Both graphs can be built interactively by sketching the UI and specifying the interaction using a dataflow visual language. Communication between the two graphs is managed at runtime by components we call Interaction Access Points. While developers can extend the toolkit by refining built-in generic mechanisms, UI designers can quickly and interactively design, prototype and test advanced user interfaces by applying the MaggLite principle: "draw it, connect it and run it".

show abstract

Integrating gesture and snapping into a user interface toolkit

Cited by 53 publications

References 19 publications

A model-based approach for gesture interfaces

A model-based approach for gesture interfaces

openSourcePACS: An Extensible Infrastructure for Medical Image Management

The MaggLite post-WIMP toolkit

Contact Info

Product

Resources

About