A robust method of detecting hand gestures using depth sensors

Wen, Yue; Hu, Chuanyan; Yu, Guangzheng; Wang, Changbo

doi:10.1109/have.2012.6374441

Cited by 48 publications

(25 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Selain itu, terdapat penelitian mengenai penggunaan tangan sebagai interaksi alamiah dengan Augmented-Reality Interfaces [3] dan penelitian mengenai pengenalan bahasa isyarat yaitu pengenalan American Sign Language (ASL) berdasarkan nilai estimasi pose tangan dari depth sensor secara real time [4]. Penggunaan Kinect dalam penelitian teah banyak dilakukan diantaranya penelitian mengenai metodeuntuk mendeteksi tangan dan gerakan tangan menggunakan depth sensor Kinect [5][6], penelitian untuk memperkirakan pose kepala secara real time menggunakan depth sensor [7], dan penelitian mengenai skema untuk mengekstraksi tangan dari depth image [8].…”

Section: Pendahuluanunclassified

Pengenalan Pose Tangan Menggunakan HuMoment

Utami¹,

Ichwan²

2017

INFOTEL

View full text Add to dashboard Cite

Abstrak -Computer vision yang didasarkan pada pengenalan bentuk memiliki banyak potensi dalam interaksi manusia dan komputer. Pose tangan dapat dijadikan simbol interaksi manusia dengan komputer seperti halnya pada penggunaan berbagai pose tangan pada bahasa isyarat. Berbagai pose tangan dapat digunakan untuk menggantikan fungsi mouse, untuk mengendalikan robot, dan sebagainya. Penelitian ini difokuskan pada pembangunan sistem pengenalan pose tangan menggunakan HuMoment. Proses pengenalan pose tangan dimulai dengan melakukan segmentasi citra masukan untuk menghasilkan citra ROI (Region of Interest), yaitu area telapak tangan. Selanjutnya, dilakukan proses deteksi tepi. Kemudian, dilakukan ekstraksi nilai HuMoment. Nilai HuMoment dikuantisasikan ke dalam bukukode yang dihasilkan dari proses pelatihan menggunakan K-Means. Proses kuantisasi dilakukan dengan menghitung nilai Euclidean Distance terkecil antara nilai HuMoment citra masukan dan bukukode. Berdasarkan hasil penelitian, nilai akurasi sistem dalam mengenali pose tangan adalah 88,57%. Various hand poses can be used to replace the mouse, motion control robots, and others. This study focused on the development of a hand pose recognition system using HuMoment quantization. The process begins with image segmentation to generate the image ROI (Region of Interest). Then the edge detection proess is performed. After that, the value of HuMoment is extracted. The HuMoment value quantized into codebook which is resulted from the training process using K-Means. Quantization process is done by calculating the smallest Euclidean Distance between the HuMoment of input image and codebook. Based on this research, the recognition accuracy is 88.57%.Keywords -Hand Pose Recognition; Computer Vision; HuMoment; Depth Image; Kinect I. PENDAHULUANDengan semakin meningkatnya penggunaan komputer dalam kehidupan sehari-hari, maka muncul tantangan baru bagaimana memberikan kenyamanan dan kemudahan dalam menggunakan komputer, salah satunya adalah kebutuhan akan metode interaksi dan komunikasi yang lebih alami antara pengguna dan computer.Penggunaan pose tangan merupakan salah satu solusi untuk memenuhi kebutuhan manusia dalam interaksi manusia dan komputer (Computer Human Interaction) yang lebih intuitif, cepat, dan sejalan dengan fungsi natural anggota tubuh manusia. Tangan dapat menghasilkan beragam pose. Setiap pose memiliki maksud dan makna tersendiri sesuai dengan kesepakatan umum ataupun kesepakatan diantara yang melakukan komunikasi. Oleh karena itu pose tangan dapat digunakan untuk menggantikan fungsi mouse, untuk mengontrol gerak robot, dan sebagainya.Depth-sensing dan gesture-tracking seperti Microsoft Kinect atau Leap Motion Controller merupakan contoh inovasi teknologi dalam metode interaksi manusia dan komputer. Teknologi tersebut memberikan peluang kepada pengguna untuk berinteraksi dengan komputer menggunakan gerakan tangan atau badan. Namun, sistem pengenalan gerakan yang digunakan berdasakan pada posisi skeleton bukan pose atau bentuk.

show abstract

Section: Pendahuluanunclassified

Pengenalan Pose Tangan Menggunakan HuMoment

Utami¹,

Ichwan²

2017

INFOTEL

View full text Add to dashboard Cite

show abstract

“…Improved results can be achieved when fusing Stereo Cameras with Depth Sensors, e.g. in [4]. In [5] a single ToF-Sensor is used to detect hand postures with the Viewpoint Feature Histogram.…”

Section: Related Workmentioning

confidence: 99%

“…Subsequently, in Sec. 4 we will give an account of the used different holistic point cloud descriptors and explain the meaning of the parameter variations we will test. The key questions we will investigate in Sec.…”

Section: Introductionmentioning

confidence: 99%

Neural Network Based Data Fusion for Hand Pose Recognition with Multiple ToF Sensors

Kopinski

Gepperth

Geisler

et al. 2014

Artificial Neural Networks and Machine Learning – ICANN 2014

View full text Add to dashboard Cite

Abstract. We present a study on 3D based hand pose recognition using a new generation of low-cost time-of-flight(ToF) sensors intended for outdoor use in automotive human-machine interaction. As signal quality is impaired compared to Kinect-type sensors, we study several ways to improve performance when a large number of gesture classes is involved. We investigate the performance of different 3D descriptors, as well as the fusion of two ToF sensor streams. By basing a data fusion strategy on the fact that multilayer perceptrons can produce normalized confidences individually for each class, and similarly by designing information-theoretic online measures for assessing confidences of decisions, we show that appropriately chosen fusion strategies can improve overall performance to a very satisfactory level. Real-time capability is retained as the used 3D descriptors, the fusion strategy as well as the online confidence measures are computationally efficient.

show abstract

“…Improved results can be achieved when fusing Stereo Cameras with Depth Sensors, e.g. in [5]. In [6] a single ToF-Sensor is used to detect hand postures with the Viewpoint Feature Histogram.…”

Section: Related Workmentioning

confidence: 99%

A real-time applicable 3D gesture recognition system for automobile HMI

Kopinski

Geisler

Caron

et al. 2014

17th International IEEE Conference on Intelligent Transportation Systems (ITSC)

View full text Add to dashboard Cite

Abstract-We present a system for 3D hand gesture recognition based on low-cost time-of-flight(ToF) sensors intended for outdoor use in automotive human-machine interaction. As signal quality is impaired compared to Kinect-type sensors, we study several ways to improve performance when a large number of gesture classes is involved. Our system fuses data coming from two ToF sensors which is used to build up a large database and subsequently train a multilayer perceptron (MLP). We demonstrate that we are able to reliably classify a set of ten hand gestures in real-time and describe the setup of the system, the utilised methods as well as possible application scenarios. I. INTRODUCTIONAs "intelligent" devices enter more and more areas of everyday life, the issue of man-machine interaction becomes ever more important. As interaction should be easy and natural for the user and also not require a high cognitive load, non-verbal means of interaction such as hand gestures will play a decisive role in this field of research. With the advent of low-cost Kinect-type 3D sensors, and more recently of low-cost ToF sensors (400-500e) that can be applied in outdoor scenarios, the use of point clouds seems a very logical choice. This presents challenges to machine learning approaches as the data dimensionality and sensor noise are high, as well as the number of interesting gesture categories. In this article we build upon earlier results [1] and demonstrate how a system can be developed and integrated into a car in order to be able to classify a gesture alphabet of ten hand poses. Our approach is purely data-driven, i.e. by extending and applying a Pointcloud descriptor to our needs we are able to set up a real-time applicable system which is robust versus daylight interferences, invariant to rotation and translation problems and moreover works without the need to formalise a possibly complicated hand model. We will first discuss the related work relevant for our research (Sec. II) and then go on to describe the setup of our system within an automobile environment. Subsequently we describe the sensors and the used database in Sec. IV. In Sec. V we go on to give an account of the used different holistic point cloud descriptors and explain the meaning of the parameter variations we will test. Sec. VI summarises the implemented NN classes and the choice of parameters. The key questions we will investigate in Sec. VII concern the generalisation

show abstract

A robust method of detecting hand gestures using depth sensors

Cited by 48 publications

References 16 publications

Pengenalan Pose Tangan Menggunakan HuMoment

Pengenalan Pose Tangan Menggunakan HuMoment

Neural Network Based Data Fusion for Hand Pose Recognition with Multiple ToF Sensors

A real-time applicable 3D gesture recognition system for automobile HMI

Contact Info

Product

Resources

About