Remote Blood Oxygen Estimation From Videos Using Neural Networks

Abstract

Blood oxygen saturation (SpO 2 ) is an essential indicator of respiratory functionality and is receiving increasing attention during the COVID-19 pandemic. Clinical findings show that it is possible for COVID-19 patients to have significantly low SpO 2 before any obvious symptoms. The prevalence of cameras has motivated researchers to investigate methods for monitoring SpO 2 using videos. Most prior schemes involving smartphones are contact-based: They require a fingertip to cover the phone's camera and the nearby light source to capture re-emitted light from the illuminated tissue. In this paper, we propose the first convolutional neural network based noncontact SpO 2 estimation scheme using smartphone cameras. The scheme analyzes the videos of a participant's hand for physiological sensing, which is convenient and comfortable, and can protect their privacy and allow for keeping face masks on. We design our neural network architectures inspired by the optophysiological models for SpO 2 measurement and demonstrate the explainability by visualizing the weights for channel combination. Our proposed models outperform the state-of-theart model that is designed for contact-based SpO 2 measurement, showing the potential of our proposed method to contribute to public health. We also analyze the impact of skin type and the side of a hand on SpO 2 estimation performance.

Automated Remote Pulse Oximetry System (ARPOS)

“…This was obtained by comparing red and blue wavelength bands; however, these studies [6,28,51], only used a 0.5 m range, which is short for a realistic scenario [6,28,51]. Another study used hand palms to measure SPO 2 by applying spatial averaging, obtaining R, G, B time series and applying CNN structure; however, using palms under a camera is not very practical as people in real life would be required to keep their hands still under a camera, something that is unrealistic in healthcare settings [27].…”

“…Previous studies have used a variety of equipment and set-ups to capture data from participants to obtain HR. Thermal [52][53][54], charge-coupled device (CCD) [55,56], other affordable web cameras or those built in laptops [19,22], Microsoft Kinect V2 [57], GoPro camera with drone [58], and smartphone [27] cameras have been previously used to obtain a person's vitals. Each device has its own characteristics, including resolution, dimensions, processing power, data type collected, and cost.…”

“…Previous research data were gathered mostly in a controlled environment or lab setting [19,[22][23][24][25][26][27]33,49,51]. Studies conducted restrict participants from micro facial gestures or physical movement of their arms or face rotations.…”

Contact-Free Simultaneous Sensing of Human Heart Rate and Canine Breathing Rate for Animal Assisted Interactions

“…An interesting alternative approach is to use camera-based remote physiological monitoring methods that do not require physical contact or proprietary infrastructure for data collection. From human subjects, the seminal and subsequent research in this field has found some success in extracting skin color signals that are highly related to heart rate (HR) and blood oxygenation, in stationary or even in substantial motion use cases [6,22,25,34,42]. In this vein, extensive human remote physiological monitoring work has been done in the literature to detect HR in physical exercising contexts [42] and extracting a blood oxygenation signal from the hand [25,34].…”

“…From human subjects, the seminal and subsequent research in this field has found some success in extracting skin color signals that are highly related to heart rate (HR) and blood oxygenation, in stationary or even in substantial motion use cases [6,22,25,34,42]. In this vein, extensive human remote physiological monitoring work has been done in the literature to detect HR in physical exercising contexts [42] and extracting a blood oxygenation signal from the hand [25,34]. The animal remote physiological monitoring landscape is much more limited with most work being done in dogs using radar or infrared devices rather than regular redgreen-blue (RGB) cameras [29,[36][37][38], or in computer vision detection of physiological signals in cattle, pigs, and exotic animals [2,4,17,18,24,27,35].…”