Solar-Powered Deep Learning-Based Recognition System of Daily Used Objects and Human Faces for Assistance of the Visually Impaired

Abstract

This paper introduces a novel low-cost solar-powered wearable assistive technology (AT) device, whose aim is to provide continuous, real-time object recognition to ease the finding of the objects for visually impaired (VI) people in daily life. The system consists of three major components: a miniature low-cost camera, a system on module (SoM) computing unit, and an ultrasonic sensor. The first is worn on the user’s eyeglasses and acquires real-time video of the nearby space. The second is worn as a belt and runs deep learning-based methods and spatial algorithms which process the video coming from the camera performing objects’ detection and recognition. The third assists on positioning the objects found in the surrounding space. The developed device provides audible descriptive sentences as feedback to the user involving the objects recognized and their position referenced to the user gaze. After a proper power consumption analysis, a wearable solar harvesting system, integrated with the developed AT device, has been designed and tested to extend the energy autonomy in the different operating modes and scenarios. Experimental results obtained with the developed low-cost AT device have demonstrated an accurate and reliable real-time object identification with an 86% correct recognition rate and 215 ms average time interval (in case of high-speed SoM operating mode) for the image processing. The proposed system is capable of recognizing the 91 objects offered by the Microsoft Common Objects in Context (COCO) dataset plus several custom objects and human faces. In addition, a simple and scalable methodology for using image datasets and training of Convolutional Neural Networks (CNNs) is introduced to add objects to the system and increase its repertory. It is also demonstrated that comprehensive trainings involving 100 images per targeted object achieve 89% recognition rates, while fast trainings with only 12 images achieve acceptable recognition rates of 55%.

Wearable devices and IoT applications for symptom detection, infection tracking, and diffusion containment of the COVID-19 pandemic: a survey

“…are employed, using properly defined metrics, to determine and track the duration and intensity of the social contact. Furthermore, several wearable IoT solutions have been proposed to remotely monitor biophysical conditions and detect symptoms commonly associated with COVID-19 early (cough, shortness of breath, low SpO 2 level) (Calabrese et al, 2020;de Fazio et al, 2020b;Grant et al, 2020;Larsen et al, 2020;Seshadri et al, 2020;Visconti et al, 2020).…”

Development of a Self-Powered Piezo-Resistive Smart Insole Equipped with Low-Power BLE Connectivity for Remote Gait Monitoring

“…It is equipped with a matrix of piezoresistive sensors based on the Velostat ® layer (manufactured by 3M Electronics division, Saint Paul, MN, USA), based on a sandwich structure [ 12 , 13 ]. The main contributions of the scientific work are: A comprehensive characterization of Velostat ® -based piezoresistive sensors with different sizes, support materials, fixing methods, and pressure trends to determine the most suitable solution for implementing the sensing matrix; Design of the sensing matrix, including 8 FSRs with size 3 × 1 cm 2 , interfaced with a conditioning and acquisition section based on Arduino Lilypad board; Testing of the low-power smart insole by acquiring pressure and acceleration data provided by the sensing matrix and 3-axis accelerometer; the insole includes a piezoelectric harvesting section to scavenge energy from the user walking [ 8 , 14 ]; Development of a custom Processing ® application, implementing an interpolation method for extending the acquired pressure map. …”

“…Testing of the low-power smart insole by acquiring pressure and acceleration data provided by the sensing matrix and 3-axis accelerometer; the insole includes a piezoelectric harvesting section to scavenge energy from the user walking [ 8 , 14 ];…”

An Overview of Wearable Piezoresistive and Inertial Sensors for Respiration Rate Monitoring

“…Wearable sensors are becoming increasingly popular, allowing continuous and unobtrusive monitoring of many biophysical and environmental parameters for different applications and operative scenarios, thus improving users' life quality and preventing diseases [1,2]. In this field, the Internet of Things (IoT) represents a milestone on which the future health system will be built, combining wearable sensing devices with cloud and fog computing, predictive and inferring techniques [3,4].…”