Augmented assembly using a mobile phone

Abstract

We present a mobile phone based augmented reality (AR) assembly system that enable users to view complex models on their mobile phones. It is based on a client-server architecture, where complex model information is located on a PC, and a mobile phone with the camera is used as a thin client access device to this information. With this system users are able to see an AR view that provides step by step guidance for a real world assembly task. We also present results from a pilot user study evaluating the system, showing that people felt the interface was intuitive and very helpful in supporting the assembly task. INTRODUCTIONWe are interested in exploring how augmented reality (AR) technology can be used to provide intuitive assistance with real world assembly, repair and training tasks. In this paper we present our mobile phone based augmented reality assembly system. We first review previous related work, and then describe our prototype based on a mobile phone client and remote PC server architecture. Finally we provide some preliminary results from a pilot user study evaluating the system. RELATED WORKSeveral other research groups have explored the use of AR for assisting with real world assembly tasks. One of the earliest of these efforts was augmenting of wire harness assembly at Boeing [1], using a wearable PC with head mounted display attached. More recent augmented assembly and maintenance solutions have been based on portable PC hardware, for example BMW's augmented car maintenance system [3]. Among the first to employ handheld devices were Geiger et al. [2], in an application providing augmented operation instructions for home appliances. Currently hand held (ultra mobile) PCs are a popular choice for portable AR assembly applications, e.g. the ULTRA system [4].Most recently, mobile phones have been used for AR applications, although not for assembly tasks. The first of these, AR-Phone [5], used Bluetooth to send phone camera images to a remote sever for processing and graphics overlay, taking several seconds per image. Since then, Moehring [7] developed the first example of an AR application running entirely on the phone, while Henrysson ported the ARToolKit library over to the Symbian phone platform [6]. Since that time there have been other examples of mobile phone based AR interfaces, e.g. [8].As can be seen, AR technology has shown to be useful for assembly tasks, but until now it has not been delivered on a mobile phone platform. Our work is novel because it provides the first example of an AR assembly interface delivered on a mobile phone, it has unique interface elements that have not been demonstrated before in an AR assembly application, and it also provides a user study of the AR interface. DEMONSTRATION SYSTEMThere are a number of challenges in providing a high quality user experience for augmented assembly on mobile phones. Product model formats vary enormously, and the number of polygons can be very large (e.g the 0.5 million triangle model in Figure 1 is very small for industrial ...

High-precision vision-based mobile augmented reality system for context-aware architectural, engineering, construction and facility management (AEC/FM) applications

“…The main drawback of these Radio Frequency based (RF-based) location tracking technologies is their high degree of dependency on pre-installed infrastructure, which makes their application either difficult or impractical for construction sites. Application of fiducial markers is also suggested by several researchers (e.g., Feng and Kamat 2012;Hakkarainen et al 2008;Lee and Akin 2011;Yakubi et al 2011). The systems are also infrastructure-dependent and require the markers to be attached to various surfaces on construction sites, which challenges their applications for large-scale implementations.…”

“…To exploit these benefits, many research projects have focused on providing cyber-information to field personnel through mobile devices and/or augmented reality systems (e.g., Akula et al 2011;Anumba and Aziz 2006;Behzadan et al 2008;Shin and Dunston 2008;Hakkarainen et al 2008;Khoury and Kamat 2009;Irizarry et al 2012;Pasman and Woodward 2003). These works have primarily focused on using Global Positioning Systems (GPS), Wireless Local Area Networks (WLAN), or Indoor GPS for accurately positioning the user within congested construction environments.…”

Fast and scalable structure-from-motion based localization for high-precision mobile augmented reality systems

“…In the meantime, several research groups have proposed marker-based mobile augmented reality to remove the dependency on mobile sensors or pre-installed network infrastructures [11][12][13][14][15][16]. These works track users' position and orientation using image processing techniques, i.e., matching the image captured by users' mobile devices to special, pre-defined 2D patterns (markers).…”

Interactive Assembly Guide Using Augmented Reality

“…AR is also seen in for example computer games [4] and mobile social events [5]. Lastly AR is also seen in assembly, inspection, and maintenance, where the overlayed graphics ease the task of the operator by providing supporting information [6]. The pose of the object to be augmented needs to be known.…”

“…This is especially problematic when AR is used on see-though displays and the result ranges from mere irritation to actual motion sickness [8]. To avoid these problems dierent systems use alternative ways to estimate the pose of the object: GPS [9], known transformation between the camera and the real environment as seen in AR sport applications [6], or pre-made patterns are attached to the object to be pose estimated, like seen in ARToolKit [10].…”