Visual Keyboard for Smart TV Using RGB Camera

Abstract

This paper proposes an effective and robust hand tracking and fingertip detection method and applies it to a novel vision based human computer interaction system: Visual Keyboard System For Smart TV (VKS). The both hands tracking and fingertip recognition approach consists of three stages. First, based on the CAMSHIFT, Motion Template and Shape Template Matching, the both hands were tracked by Multi Object Tracking CAMSHIFT. Then, the location of fingertip was localized precisely based on Motion Template. After those process, we have the Robust Shape Template Matching to recognize a fingertip. Experiments suggest that the proposed our method is capable of detecting fingertip recognition in a reliable manner even in a complex background under different light conditions without any markers. The Fingertip Recognition Visual Keyboard System in this paper is particularly advantageous for Human-computer Interaction (HCI) in that users can communicate with SMART TV by their favorite mean: fingertip gesturing. At the same time, they can perform typing with only their fingertip directly.