A method for real-time generation of augmented reality work instructions via expert movements

Abstract

Augmented Reality (AR) offers tremendous potential for a wide range of fields including entertainment, medicine, and engineering. AR allows digital models to be integrated with a real scene (typically viewed through a video camera) to provide useful information in a variety of contexts. The difficulty in authoring and modifying scenes is one of the biggest obstacles to widespread adoption of AR. 3D models must be created, textured, oriented and positioned to create the complex overlays viewed by a user. This often requires using multiple software packages in addition to performing model format conversions. In this paper, a new authoring tool is presented which uses a novel method to capture product assembly steps performed by a user with a depth+RGB camera. Through a combination of computer vision and imaging process techniques, each individual step is decomposed into objects and actions. The objects are matched to those in a predetermined geometry library and the actions turned into animated assembly steps. The subsequent instruction set is then generated with minimal user input. A proof of concept is presented to establish the method's viability.

Furthering Service 4.0: Harnessing Intelligent Immersive Environments and Systems

“…Initiatives such as the IEEE Smart Glasses Roadmap Group [61] are working to promote and facilitate the adoption of smart glasses in several markets and applications, and it is likely that technology developments will soon address the limitations. Another reason that the implementation of intelligent immersive systems remains limited is in large part due to the expense associated with creating and changing content to improve procedures in everyday business operations [62], [63]. Content creation software is limited, as content is usually not generated in real time, and it lacks flexibility and ease of integration.…”

Using Freeze Frame and Visual Notifications in an Annotation Drawing Interface for Remote Collaboration

“…With Augmented Reality (AR) technology, researchers can add virtual objects in the real world to present context-sensitive information [1]. One of the main purpose of it is providing information to solve a physical task, especially for supporting training [2,3,4,5,6,7,8] and improving remote collaboration [9]. In this paper, we study real-time remote collaboration between local and remote users, and how AR visual cues can be used to improve collaboration.…”

Automatic generation of augmented reality guided assembly instructions using expert demonstration

“…there is no need to understand complicated algorithms as all the required parameters are set automatically and can be treated as a black box, and 3) PSOGMM is faster than GMM in training by 97% and by 49% in testing while comparable in precision and recall. For complete details on the algorithm, please see [243].…”

“…In the previous version of this algorithm [243] finding the number of steps was based on a threshold value 'hand/non-hand' for the number of skin pixels that constitute a hand. Getting the appropriate 'hand/non-hand' threshold was a matter of trial and error and constantly changed with different input parameters.…”

“…Demonstration of handCurrent popular skin detection techniques like Gaussian mixture models (GMM) are lacking because: 1) they are not designed for unclean data, 2) they require the author to understand complicated algorithms to set the parameters appropriately, and 3) complicated computer science concepts like overfitting cannot be easily understood. Hence, a new algorithm known as particle swarm Optimized Gaussian mixture model (PSOGMM) was developed[243].…”