Context-Aware Online Adaptation of Mixed Reality Interfaces

Currently, new technologies have enabled the design of smart applications that are used as decision-making tools in the problems of daily life. The key issue in designing such an application is the increasing level of user interaction. Mixed reality (MR) is an emerging technology that deals with maximum user interaction in the real world compared to other similar technologies. Developing an MR application is complicated, and depends on the different components that have been addressed in previous literature. In addition to the extraction of such components, a comprehensive study that presents a generic framework comprising all components required to develop MR applications needs to be performed. This review studies intensive research to obtain a comprehensive framework for MR applications. The suggested framework comprises five layers: the first layer considers system components; the second and third layers focus on architectural issues for component integration; the fourth layer is the application layer that executes the architecture; and the fifth layer is the user interface layer that enables user interaction. The merits of this study are as follows: this review can act as a proper resource for MR basic concepts, and it introduces MR development steps and analytical models, a simulation toolkit, system types, and architecture types, in addition to practical issues for stakeholders such as considering MR different domains.

show abstract

“…Ref. [116] offered a combination of rule-based decision-making and optimization for UI adaption using virtual element LOD and cognitive load.…”

Section: Ui Issuementioning

confidence: 99%

“…FOV influences how much information is displayed. Cognitive load estimation requires techniques that utilize sensor fusion to handle sudden interruption and errors during this estimation [116].…”

Section: Ui Issuementioning

confidence: 99%

A Review on Mixed Reality: Current Trends, Challenges and Prospects

2020

View full text Add to dashboard Cite

show abstract

“…For example, a design can be reoptimized when the device orientation changes [85]. Todi et al [79] adapted web layouts in line with the visual history of a user [80], and Lindlbauer et al [86] proposed an algorithm to automatically adapt mixed reality interfaces in line with the current user context, leveraging the environment and the cognitive load imposed by the user's current task. In real time, their optimization-based approach adjusts which information is displayed in the interface and where it gets placed.…”

Section: Applications 1) Generative Designmentioning

confidence: 99%

Combinatorial Optimization of Graphical User Interface Designs

et al. 2020

View full text Add to dashboard Cite

show abstract

“…Other systems extract relevant information to compile a summary for later use [Buscher et al 2012a;Nguyen and Liu 2016]. UI adaptation based on user's cognitive load or context has recently been shown for mixed-reality settings [Gebhardt et al 2019;Lindlbauer et al 2019] where the design of a good UI can be difficult as the user's context frequently changes, similar to other applications of ubiquitous computing [Dourish 2004].…”

Section: Ui Adaptation From Gaze Behaviormentioning

confidence: 99%

Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation

Feit

Vordemann

Park

et al. 2020

ACM Symposium on Eye Tracking Research and Applications

Self Cite

View full text Add to dashboard Cite

This paper proposes an approach to detect information relevance during decision-making from eye movements in order to enable user interface adaptation. This is a challenging task because gaze behavior varies greatly across individual users and tasks and groundtruth data is difficult to obtain. Thus, prior work has mostly focused on simpler target-search tasks or on establishing general interest, where gaze behavior is less complex. From the literature, we identify six metrics that capture different aspects of the gaze behavior during decision-making and combine them in a voting scheme. We empirically show, that this accounts for the large variations in gaze behavior and out-performs standalone metrics. Importantly, it offers an intuitive way to control the amount of detected information, which is crucial for different UI adaptation schemes to succeed. We show the applicability of our approach by developing a room-search application that changes the visual saliency of content detected as relevant. In an empirical study, we show that it detects up to 97% of relevant elements with respect to user self-reporting, which allows us to meaningfully adapt the interface, as confirmed by participants. Our approach is fast, does not need any explicit user input and can be applied independent of task and user. CCS CONCEPTS • Human-centered computing → HCI theory, concepts and models; Empirical studies in HCI.

show abstract

Context-Aware Online Adaptation of Mixed Reality Interfaces

Cited by 124 publications

References 44 publications

A Review on Mixed Reality: Current Trends, Challenges and Prospects

A Review on Mixed Reality: Current Trends, Challenges and Prospects

Combinatorial Optimization of Graphical User Interface Designs

Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation

Contact Info

Product

Resources

About