Wearable Reading Assist System: Augmented Reality Document Combining Document Retrieval and Eye Tracking

Toyama, Takumi; Dengel, Andreas; Suzuki, Wakana; Kise, Koichi

doi:10.1109/icdar.2013.15

Cited by 23 publications

(10 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Previous work shows that activation of commands using dwell-time-based approaches are useful, and can even detect fixations with noisy data [26]. We implement this function by detecting attention to icons on the display.…”

Section: Eye-conmentioning

confidence: 97%

“…Recent work by Coco et al [8] and Henderson et al [10] also showed that further classification of cognitive state such as scene memorization and visual search is possible using eye gaze patterns. Such analysis on user's cognitive state could be used to enhance user experiences when he or she is interacting with a computer screen [20,5] or with real document printouts [26], inferring in which type of cognitive state the user is involved.…”

Section: Prior Workmentioning

confidence: 99%

“…Though several eye gaze-based interactive AR systems have already been proposed [13,26], there are still only a handful of interactions available through gaze, and many of them are limited to line of sight. Usually, line of sight-based techniques have an inevitable drawback when used with optical see-through displays in that they cannot distinguish on which interactive space (virtual or real) the user is focused.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions

Toyama

Sonntag

Orlosky

et al. 2015

Proceedings of the 20th International Conference on Intelligent User Interfaces

Self Cite

View full text Add to dashboard Cite

Human eye gaze has recently been used as an effective input interface for wearable displays. In this paper, we propose a gaze-based interaction framework for optical see-through displays. The proposed system can automatically judge whether a user is engaged with virtual content in the display or focused on the real environment and can determine his or her cognitive state. With these analytic capacities, we implement several proactive system functions including adaptive brightness, scrolling, messaging, notification, and highlighting, which would otherwise require manual interaction. The goal is to manage the relationship between virtual and real, creating a more cohesive and seamless experience for the user. We conduct user experiments including attention engagement and cognitive state analysis, such as reading detection and gaze position estimation in a wearable display towards the design of augmented reality text display applications. The results from the experiments show robustness of the attention engagement and cognitive state analysis methods. A majority of the experiment participants (8/12) stated the proactive system functions are beneficial.

show abstract

Section: Eye-conmentioning

confidence: 97%

Section: Prior Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions

Toyama

Sonntag

Orlosky

et al. 2015

Proceedings of the 20th International Conference on Intelligent User Interfaces

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although focus has previously been proposed for interaction with text such as the system proposed by Toyama et al, research up until now lacks interaction methods for multi-focal plane HMDs [22]. Despite the appearance of several multi-focal or vari-focal HMDs, studies with those displays are limited to depth perception and have yet to take advantage of focal depth via eye tracking.…”

Section: Gaze Based Interaction In Augmented Realitymentioning

confidence: 99%

A natural interface for multi-focal plane head mounted displays using 3D gaze

Toyama

Orlosky

Sonntag

et al. 2014

Proceedings of the 2014 International Working Conference on Advanced Visual Interfaces

Self Cite

View full text Add to dashboard Cite

In mobile augmented reality (AR), it is important to develop interfaces for wearable displays that not only reduce distraction, but that can be used quickly and in a natural manner. In this paper, we propose a focal-plane based interaction approach with several advantages over traditional methods designed for head mounted displays (HMDs) with only one focal plane. Using a novel prototype that combines a monoscopic multi-focal plane HMD and eye tracker, we facilitate interaction with virtual elements such as text or buttons by measuring eye convergence on objects at different depths. This can prevent virtual information from being unnecessarily overlaid onto real world objects that are at a different range, but in the same line of sight. We then use our prototype in a series of experiments testing the feasibility of interaction. Despite only being presented with monocular depth cues, users have the ability to correctly select virtual icons in near, mid, and far planes in 98.6% of cases.

show abstract

“…This method, called LLAH (Locally Likely Arrangement Hashing) is robust to perspective distortion of an image and scale-invariant. The method has been used by [11]. An overview of the document retrieval method is shown in Figure 3.…”

Section: Document Retrievalmentioning

confidence: 99%

Analysis of text layout quality using wearable eye trackers

Chanijani

Bukhari

Dengel

2015

2015 IEEE International Conference on Multimedia &Amp; Expo Workshops (ICMEW)

Self Cite

View full text Add to dashboard Cite

How would considering the aesthetics of text layout, particularly line spacing with the Golden Ratio (GR) parameter, influence the ease of reading and retention? To answer this question we introduced a novel method by employing a wearable eye tracker instead of a table-mounted eye tracker. Because of their lower price, portability, and growing market, as well as convenience to setup, such head-mounted devices are suitable use for ubiquitous reading behavior analysis. For mapping gaze information from captured video scenes to the original document, we adopted the Locally Likely Arrangement Hashing (LLAH) method for robust document retrieval. In our experimental system, participants read digital documents from the screen with and without the aesthetic GR parameter. Then, gaze data captured by the eye tracker's embedded camera is mapped to the corresponding original document. Finally, our gaze analysis system extracts the intended information for statistical evaluation. Regarding to the results, significant differences in reading performance were found in the documents with and without the GR aesthetic parameter for line spacing.

show abstract

Wearable Reading Assist System: Augmented Reality Document Combining Document Retrieval and Eye Tracking

Cited by 23 publications

References 10 publications

Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions

Attention Engagement and Cognitive State Analysis for Augmented Reality Text Display Functions

A natural interface for multi-focal plane head mounted displays using 3D gaze

Analysis of text layout quality using wearable eye trackers

Contact Info

Product

Resources

About