Designing Gesture-Based Control for Factory Automation

Abstract

Abstract. We report the development and evaluation of a gesture-based interaction prototype for controlling the loading station of a factory automation system. In this context, gesture-based interaction has the potential to free users from the tedious physical controls but it must also account for safety considerations and users' perceptions. We evaluated the gesture interaction concept in the field to understand its applicability to industrial settings. Our findings suggest that gesture-based interaction is an emotional, physically charged experience that has the potential to enhance the work process. Participants' feedback also highlighted challenges related to the reliability of gesture recognition technology in the workplace, the perceived professionalism of gesture-based interaction, and the role of physical feedback in promoting feeling of control. Our results inform the development of gesture-based interaction for similar contexts.Keywords: Gesture-based interaction, emotions, user experience, field study. IntroductionGesture-based interaction could provide many benefits as an alternative to physical controls in industrial settings, where the manual operation of machinery can be ergonomically challenging and hazardous or is governed by safety regulations. Several characteristics of gesture-based systems affect their usability in such settings, including technical aspects such as latency of operation, stability, resolution, and precision [1], as well as human-centered aspects such as intuitiveness, ergonomics, and ease of gesture performance and recall [2]. Appropriately designed systems must balance these sometimes contradictory concerns. We report the findings from a year-long design and development process of a gesture-based interaction concept, focusing on the results of evaluations conducted in real context of use. The overall goal of the research was to create novel interaction concepts for metal workshops and to study their potential in real work tasks. Our research was aimed at answering the following questions:

Emotional Dimensions of User Experience: A User Psychological Analysis

“…Participants were asked to perform gestures in front of a loading station simulator, which consisted of a large computer screen (40 inches) and a gesture detector (Microsoft Kinect). The simulator was created in Tampere Unit for Computer-Human Interaction (TAUCHI), and its specifics are reported by Heimonen et al (2013). Gesture-based interaction was a novel concept in the factory environment, and the participants had no experience of using gestures to control factory automation.…”

From a literature review to a conceptual framework of enablers for smart manufacturing control

“…An example of modern ways of HMI is automatic speech recognition [119]. Another example is visual gesture control using expressive and meaningful body motions [123,128]. Further, we can have physical human-robot interaction as, under the perspective of the multimodal interface, physical contact may be used to create an interaction interface for CPS.…”

Effects of auditory, haptic and visual feedback on performing gestures by gaze or by hand

“…Nowadays, hand gestures are usually dynamic, which means they do not consist of static pointing only (Köpsel and Huckauf 2013;Villamor, Willis, and Wroblewski 2010;Wobbrock, Morris, and Wilson 2009). Gestures have been used for controlling smartphones, home electronics (Lenman, Bretzner, and Thuresson 2002;Shan 2010), factory automation (Heimonen et al 2013), humanrobot interaction (Alvarez-Santos et al 2014), multitouch surfaces (Wobbrock, Morris, and Wilson 2009) and many other electronic devices (Baudel and Beaudouin-Lafon 1993;Bhuiyan and Picking 2009;Garzotto and Valoriani 2012), even head-up displays (Saxen et al 2012). Whereas earlier systems needed utilities such as gloves (Baudel and Beaudouin-Lafon 1993) or other tracking targets (Tsukadaa and Yasumura 2002;Zimmerman et al 1987), these days no equipment attached to the body is necessary (Shan 2010).…”