FingerReader

Abstract

Accessing printed text in a mobile context is a major challenge for the blind. A preliminary study with blind people reveals numerous difficulties with existing state-of-the-art technologies including problems with alignment, focus, accuracy, mobility and efficiency. In this paper, we present a finger-worn device, FingerReader, that assists blind users with reading printed text on the go. We introduce a novel computer vision algorithm for local-sequential text scanning that enables reading single lines, blocks of text or skimming the text with complementary, multimodal feedback. This system is implemented in a small finger-worn form factor, that enables a more manageable eyes-free operation with trivial setup. We offer findings from three studies performed to determine the usability of the FingerReader.

Next Steps for Human-Computer Integration

“…Fusion might also occur with implanted devices [28,84], ingested devices [41,81], or epidermal electronics [80], as well as devices which extend or manipulate the body (e.g. [86,74]), or stimulate the senses [47,73,83,98].…”

Assistive Technologies for Individuals With Visual Impairment: Contribution of Design in the Field of Clothing

“…Wearers scan a text line with their finger and receive audio feedback of the words and a haptic feed. In addition, Finger Reader's software stack includes a sequential text-reading algorithm, hardware control driver, integration layer with Tesseract OCR and Flite Text-to-Speech feed (Shilkrot, 2015).…”

Design of an Augmented Reality Magnification Aid for Low Vision Users

“…Other systems use cameras worn on the head [11] or wrist [19] to identify faces, and an early version of Microsoft's recent Seeing AI smartphone app [37] used a pair of smart-glasses to identify people, describe facial attributes, and read text, and was controlled by touch gestures on the side of the glasses [38]. Finger-mounted cameras have also been used to support visually impaired users in a variety of tasks, including reading printed text via text-to-speech output [18,21], controlling mobile devices [22], and identifying currency or other objects [13].…”

“… Content capture. To capture video for processing and display, possible camera locations include head-worn (e.g., [14,25,[27][28][29]), hand-held (e.g., [33,34,37]), and finger or wrist-worn (e.g., ring or smartwatch; [18][19][20]). We explore a few of these options and discuss the advantages and disadvantages of each.…”

“…The camera provided a close-up view of the target content and allowed the user to read without needing to frame the text within the head-worn camera's field of view. As discussed earlier, similar wearable cameras have been used for other assistive devices [13,18,21,22], albeit primarily for people with more severe visual impairments. We used a laptop computer to capture and process images from the camera, which we streamed wirelessly in real-time to the HoloLens for display.…”