Task support system by displaying instructional video onto AR workspace

Abstract

This paper presents an instructional support system based on augmented reality (AR). This system helps a user to work intuitively by overlaying visual information in the same way of a navigation system. In usual AR systems, the contents to be overlaid onto real space are created with 3D Computer Graphics. In most cases, such contents are newly created according to applications. However, there are many 2D videos that show how to take apart or build electric appliances and PCs, how to cook, etc. Therefore, our system employs such existing 2D videos as instructional videos. By transforming an instructional video to display, according to the user's view, and by overlaying the video onto the user's view space, the proposed system intuitively provides the user with visual guidance. In order to avoid the problem that the display of the instructional video and the user's view may be visually confused, we add various visual effects to the instructional video, such as transparency and enhancement of contours. By dividing the instructional video into sections according to the operations to be carried out in order to complete a certain task, we ensure that the user can interactively move to the next step in the instructional video after a certain operation is completed. Therefore, the user can carry on with the task at his/her own pace. In the usability test, users evaluated the use of the instructional video in our system through two tasks: a task involving building blocks and an origami task. As a result, we found that a user's visibility improves when the instructional video is transformed to display according to his/her view. Further, for the evaluation of visual effects, we can classify these effects according to the task and obtain the guideline for the use of our system as an instructional support system for performing various other tasks.

A Mobile Solution for Augmenting a Manufacturing Environment with User-Generated Annotations

“…Textual information with annotations and simple instructive arrows are often used due to the limitation of the processing capability of hardware. Some researchers have proposed to overlay previously recorded videos directly onto the AR workspace [8]. Haptic and acoustic feedback may also be added to aid the users with detailed explanations and alert the users to potential hazardous situations [9].…”

Poster: 3D referencing for remote task assistance in augmented reality

“…Kirk and Fraser directly embedded the gestural output of an expert in the local view of a technician captured from live overhead video [6], while Goto et al presented a similar technique using prerecorded video [3]. These techniques may work well on tasks performed on a flat surface, but could be challenging to apply to more complex environments.…”

TutoriVR

“…Video is a lightweight format for capturing workfows. Several AR systems use video to deliver physical assembly instructions [12,16,30]. In contrast, our focus is on helping users create virtual content in VR.…”