Object manipulation and deformation using hand gestures

Abstract

Compared to the mouse, as a two-dimensional and precise interface device, hand gestures provide more degrees of freedom for users to interact with computers by employing intelligent computing methods. Leap Motion Controller is gaining more popularity due to its ability to detect and track hand joints in three dimensions. However, in some cases, the Leap Motion Controller measurements are not correct enough. We show that the occlusion, palm angle, and the limited field of view are the main downsides of the Leap Motion Controller. In this paper, a framework is proposed to manipulate and deform a three-dimensional object by hand gestures. We select only a few gestures so that the system instructions can be easily memorized. The gestures are not defined very strictly, so users can do them properly without getting tired. We propose that calculating a reliable space from a Leap Motion Controller can significantly reduce these problems. To deform objects, the Free Form Deformation technique is used, which allows for more local deformation. The selected gestures and determined space for interaction make the deformation framework achieve a balance between the accuracy, user-factors, required tasks for deformation, and limitation of the hand tracking device. The proposed method, compared to related studies, offers more creative methods for deforming objects and more natural movements to interact with the system. According to the conducted user study, a significant difference is observed between hand gesture interaction and mouse in terms of speed and number of attempts.