Kinect cane: Object recognition aids for the visually impaired

Abstract

This paper proposes a novel concept for helping the visually impaired know what kind of object there is in an environment. This concept is implemented as a cane system that selects a target object based on a user's demand, recognizes the object from depth data obtained by a Microsoft Kinect sensor, and returns the recognition results via a tactile device. The proposed system is evaluated through a user study where one blindfolded subject actually uses the system to find chairs in an experimental environment. The experimental results indicate that the system is promising as means of helping the visually impaired recognize objects.

A review of assistive spatial orientation and navigation technologies for the visually impaired

“…Using feature descriptors and machine learning techniques, different objects can be extracted and classified [86]. These types of systems can recognize threedimensional objects from the depth data, and inform visually impaired users not only about the existence of objects but also their class, such as chairs and upward stairs [87,7,88], working similarly to a conventional white cane, with an extended range. Some systems even incorporate the detection and distinction between fixed and moving obstacles and object recognition in one global solution, mostly for pedestrian detection and avoidance [89][90][91][92].…”

“…According to [105] "one major issue to be considered in the design of an interface is whether a rich description of the scene, or only highly symbolic information, should be provided to the user". Another approach is to present the information about the obstacles detected in the image through the use of haptic interfaces [106,75,7]. 3D range data may be converted into a 2D vibrating array attached to the user's body [63].…”

“…However, it is difficult to provide a sufficient number of guide dogs because of the long-time periods needed to train them, as well as the high costs associated with their training. Furthermore, it is also quite difficult for the visually impaired to take care of the living dogs appropriately [7]. Modern mobile assistive technologies are becoming more discrete and include (or are delivered via) a wide range of mobile computerized devices, including ubiquitous technologies like mobile phones.…”

Indoor Positioning System Using Depth Maps and Wireless Networks

“…Ye et al [15] propose to use three Kinect sensors for detecting and identifying several people that are occluded by others in a scene. In [16], authors propose a smart-cane for the visually impaired that, with the help of a Kinect sensor, allows for locating objects. The method Kinect Positioning System (KPS) is analyzed in [17] aiming to obtain the user position.…”

Learning from depth sensor data using inductive logic programming

“…Lately smart-phone cameras [5] have also been included in the setup. Since cameras can provide information rich signals they are often used in systems for visually impaired (for navigation and obstacle avoidance) [26], [30], [28]. Unfortunately, large amount of data is at the same time a problem since much of it can not be used for intended purpose.…”