Your smartphone could act as a pulse-oximeter and as a single-lead ECG

Abstract

In the post-covid19 era, every new wave of the pandemic causes an increased concern/interest among the masses to learnmore about their state of well-being. Therefore, it is the need of the hour to come up with ubiquitous, low-cost, non-invasivetools for rapid and continuous monitoring of body vitals that reflect the status of one’s overall health. In this backdrop, this workproposes a deep learning approach to turn a smartphone—the popular hand-held personal gadget—into a diagnostic tool tomeasure/monitor the three most important body vitals, i.e., pulse rate (PR), blood oxygen saturation level (aka SpO2), andrespiratory rate (RR). Furthermore, we propose another method that could extract a single-lead electrocardiograph (ECG)of the subject. The proposed methods include the following core steps: subject records a small video of his/her fingertip byplacing his/her finger on the rear camera of the smartphone, and the recorded video is pre-processed to extract the filteredand/or detrended video-photoplethysmography (vPPG) signal, which is then fed to custom-built convolutional neural networks(CNN), which eventually spit-out the vitals (PR, SpO2, and RR) as well as a single-lead ECG of the subject. To be precise,the contribution of this paper is two-fold: 1) estimation of the three body vitals (PR, SpO2, RR) from the vPPG data usingcustom-built CNNs, vision transformer, and most importantly by CLIP model (a popular image-caption-generator model); 2) anovel discrete cosine transform+feedforward neural network-based method that translates the recorded video-PPG signal to asingle-lead ECG signal. The significance of this work is two-fold: i) it allows rapid self-testing of body vitals (e.g., self-monitoringfor covid19 symptoms), ii) it enables rapid self-acquisition of a single-lead ECG, and thus allows early detection of atrialfibrillation (abormal heart beat or arrhythmia), which in turn could enable early intervention in response to a range of cardiovascular diseases,and could help save many precious lives. Our work could help reduce the burden on healthcare facilities and could lead toreduction in health insurance costs.