A computer vision-based assistant system for the assembly of narrow cabin products

Liu, Yi; Li, Shiqi; Wang, Junfeng; Zeng, Hongmei; Lu, Jiacai

doi:10.1007/s00170-014-6274-9

Cited by 41 publications

(15 citation statements)

References 32 publications

(38 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This approach directly makes use of head-worn camera and marker-based tracking, but it is not suitable for industrial scenarios due to its physical limitation. Advanced CV monitoring methods extract 2D descriptors and recognize the assembly state by feature map matching (Radkowski and Oliver, 2013), region of interest matching (Liu et al, 2015) or image distance similarity threshold matching (Petersen et al, 2013;Petersen and Stricker, 2015). Those methods monitor the position of target object during the whole assembly process; however, they lack a clear display control scheme to update the AR content according to the monitoring results.…”

Section: Assembly State Monitoring Methodsmentioning

confidence: 99%

Synchronous AR assembly assistance and monitoring system based on ego-centric vision

Yin

Fan

Zhu

et al. 2018

View full text Add to dashboard Cite

Purpose Aiming at presenting an interaction-free assembly assistance tool, the purpose of this paper is to propose a synchronous augmented reality (AR) assembly assistance and monitoring system. The system monitors operator’s hands activity and process completeness to recognize the assembly state, then display the AR contents contextually. Design/methodology/approach An assembly behavior recognition method is proposed based on gesture recognition. An assembly completeness inspection method is proposed based on SURF feature matching. Assembly state and AR display state are solved by a novel sequential hybrid AR display control strategy. A synchronous multi-channel AR view output strategy is proposed based on QR matrix decomposition. Findings A prototype system has been developed, and case study is performed on an industrial product. Experiments are performed to verify the feasibility, efficiency and recognition accuracy of the proposed methods. Research limitations/implications The proposed system assists users to perform assembly tasks with automatic visual guidance and vision monitoring, avoiding distractions caused by redundant human–computer interactions. Practical implications All methods are integrated to work on only one head-worn device, making the proposed system portable and cheaper. The vision processing pipelines and the view output channels are reconfigurable for customization. Originality/value This paper proposes an interaction-free AR assembly assistance and monitoring system. Assembly behavior recognition and assembly completeness inspection methods are integrated to monitor the assembly state. A sequential hybrid AR display control strategy is proposed to contextually update the AR contents. A synchronous multi-channel AR view output strategy is proposed to fulfill different visualization needs.

show abstract

Section: Assembly State Monitoring Methodsmentioning

confidence: 99%

Synchronous AR assembly assistance and monitoring system based on ego-centric vision

Yin

Fan

Zhu

et al. 2018

View full text Add to dashboard Cite

show abstract

“…Exemplary reasons for the introduction are technological advancements [7], the diversity of service employees in terms of education, language, experience [15] as well as demographic change and the increasing individualization of products [17] due to an increasing customer orientation [16]. Likewise, decreasing lot sizes [1,17] up to lot size 1 [5], an increasing number of components [12] as well as the increasing pressure to perform [7] due to a growing amount of special information and high demands on the skills of factory workers, among others, are crucial reasons.…”

Section: Motivation and Related Workmentioning

confidence: 99%

Potentials of Augmented Reality – Insights into Industrial Practice

Semm

Erfurth

Uslu

2021

Innovations for Community Services

View full text Add to dashboard Cite

With emerging technologies, research is increasingly focusing on the digitalization of the working world and the resulting potentials, but also risks, for employees. People and technology are beginning to grow together in industrial environments. New technologies can support skills and improve results of the daily work, but the most important factor is still the human being. Assistance systems based on Augmented Reality (AR) are increasingly attract the attention of companies. Currently, there are not many implementation reports and applications in companies. This paper discusses the findings based on observations from industrial site inspections in the areas of training, commissioning of goods, and the manual assembly process. Inspections took place at three pioneering companies in the automotive industry, online distance selling, and the energy sector. When it comes to the use of assistance systems, the main goals of these companies that are very open to innovation are to improve efficiency and quality, e.g., minimize errors and optimize speed. Qualitative content analysis was used to evaluate the interview transcripts, the collected transcripts and notes from the guided tours, and the brief participations in these industries. The results are presented in the form of user experiences with AR addressing their advantages and disadvantages in the context of use. In addition, these were categorized into a predefined classification of influencing factors and challenges.

show abstract

“…Interaction-Analysis methods have been used to capture assembly expertise, evaluate it and transfer it to robots [83][84][85] or other human users [86][87][88]. Nevertheless, the introduction of different areas of research (CA and IA) has been progressive and based on the advancements of the previous (Authoring).…”

Section: 23operations In 'Small' Assetsmentioning

confidence: 99%

“…These techniques acquire data and analyse it automatically, whose results are then use to modify content. Most techniques [14,40,41,45,47,57,70,87] still providing these results to experts for them to update augmented content. Instead, few latest papers have achieved to automatically connect these techniques with 'automatic' Authoring.…”

Section: Interaction-analysismentioning

confidence: 99%

A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications

Amo

Erkoyuncu

Roy

et al. 2018

Computers in Industry

139

View full text Add to dashboard Cite

Augmented Reality (AR) has experienced an increasing trend in applied research in the last few years. This emerging trend is focused in content-related challenges: mainly creation (Authoring), adaptation (Context-Awareness) and improvement (Interaction-Analysis) of augmented content. Research in these techniques has enabled Academia to recognise Augmented Reality capability for knowledge transfer, either from AR systems to users or between users. But to the best of author's knowledge, there are no specific literature review in these areas, neither on their relations with AR knowledge transfer ability. Therefore, this paper aims to identify these relations through an analysis of state-of-the-art techniques in Authoring (A), Context-Awareness (CA) and Interaction-Analysis (IA) in the context of maintenance applications. In order to do so, a Systematic Literature Review (SLR) has been conducted on 74 application-relevant papers from 2012 to 2017. It comprised a thematic analysis to establish the relation between maintenance applications, research in A, CA and IA and AR knowledge transfer modes. Its results helped to classify AR maintenance-applications by technological readiness levels. They also revealed the potential of AR for users' knowledge capture, and future research required for full knowledge management capabilities. Furthermore, the SLR method proposed could be extended to correlate AR systems and applications by their knowledge management capabilities in any AR application context.

show abstract

A computer vision-based assistant system for the assembly of narrow cabin products

Cited by 41 publications

References 32 publications

Synchronous AR assembly assistance and monitoring system based on ego-centric vision

Synchronous AR assembly assistance and monitoring system based on ego-centric vision

Potentials of Augmented Reality – Insights into Industrial Practice

A systematic review of Augmented Reality content-related techniques for knowledge transfer in maintenance applications

Contact Info

Product

Resources

About