Vision Based Referee Sign Language Recognition System for the RoboCup MSL League

Abstract

In RoboCup Middle Size league (MSL) the main referee uses assisting technology, controlled by a second referee, to support him, in particular for conveying referee decisions for robot players with the help of a wireless communication system. In this paper a vision-based system is introduced, able to interpret dynamic and static gestures of the referee, thus eliminating the need for a second one. The referee's gestures are interpreted by the system and sent directly to the Referee Box, which sends the proper commands to the robots. The system is divided into four modules: a real time hand tracking and feature extraction, a SVM (Support Vector Machine) for static hand posture identification, an HMM (Hidden Markov Model) for dynamic unistroke hand gesture recognition, and a FSM (Finite State Machine) to control the various system states transitions. The experimental results showed that the system works very reliably, being able to recognize the combination of gestures and hand postures in real-time. For the hand posture recognition, with the SVM model trained with the selected features, an accuracy of 98,2% was achieved. Also, the system has many advantages over the current implemented one, like avoiding the necessity of a second referee, working on noisy environments, working on wireless jammed situations. This system is easy to implement and train and may be an inexpensive solution.

Engagement Detection Framework for Hand Gesture and Posture Recognition

“…Sign language recognition [5,6,7,8], socially assistive robotics [9,10,11], directional indication through pointing [12,13], virtual controllers, and remote controllers [14,15] are a few examples of technologies employing hand gesture and posture recognition. The main focus of this research is on pixel-based posture recognition using a single camera.…”

Generic System for Human-Computer Gesture Interaction: Applications on Sign Language Recognition and Robotic Soccer Refereeing

“…They used 4-state HMMs to train each of the 19 signs defined in their study. Trigueiros [38] used HMM's applied to a Vision-based system capable of recognizing a set of referee commands for robotic soccer games. Cooper [39] implemented an isolated sign recognition system using a 1st order Markov chain.…”

Architecture for Serious Games in Health Rehabilitation

“…Users can convey information also in the form of gestures. Trigueiros et al [33][34][35][36] uses a gestures concept for a vision-based hand gesture recognition system that can be integrated in a HCI system.…”