Algorithmic Principles of Remote PPG

Abstract

Wang, W.; den Brinker, A.C.; Stuijk, S.; de Haan, G. Published in: IEEE Transactions on Biomedical Engineering DOI:10.1109/TBME.2016.2609282Published: 01/07/2017 Document VersionAuthor's version before peer-review Please check the document version of this publication:• A submitted manuscript is the author's version of the article upon submission and before peer-review. There can be important differences between the submitted version and the official published version of record. People interested in the research are advised to contact the author for the final version of the publication, or visit the DOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and page numbers. Link to publication Citation for published version (APA):Wang, W., den Brinker, A. C., Stuijk, S., & de Haan, G. (2017). Algorithmic principles of remote-PPG. IEEE Transactions on Biomedical Engineering, 64(7), 1479-1491. DOI: 10.1109/TBME.2016.2609282 General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.• Users may download and print one copy of any publication from the public portal for the purpose of private study or research.• You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal ? Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim. Abstract-This paper introduces a mathematical model that incorporates the pertinent optical and physiological properties of skin reflections with the objective to increase our understanding of the algorithmic principles behind remote photoplethysmography (rPPG). The model is used to explain the different choices that were made in existing rPPG methods for pulse extraction. The understanding that comes from the model can be used to design robust or application-specific rPPG solutions. We illustrate this by designing an alternative rPPG method where a projection plane orthogonal to the skin-tone is used for pulse extraction. A large benchmark on the various discussed rPPG methods shows that their relative merits can indeed be understood from the proposed model.

Amplitude-selective filtering for remote-PPG

“…We compared three filtering methods, i.e., Band-Pass Filter (BPF), Amplitude-Selective Filter (ASF), and ASF + BPF, as the pre-processing step in eight existing rPPG algorithms, i.e., G [1,13], G-R [14], HUE [15], PCA [2], ICA [3], CHROM [5], PBV [6], and POS [7]. The baseline for each rPPG algorithm is the bare version (None) without pre-processing, i.e., only the core algorithm for pulse extraction is addressed.…”

“…Note that an alternative to (4) is to take the logarithm of C i and remove the mean, which for small variations as the PPG-signal has practically the same effect [7]. Here we mention that C i is a 1D color-signal averaged of the skin-pixel values from a whole face, where the spatial distribution of the pulsatile signals is eliminated.…”

“…These include: (i) BSS-based approaches (PCA [2] and ICA [3]), which use different criteria to de-mix temporal RGB traces into uncorrelated or independent signal sources to retrieve the pulse; (ii) a data-driven approach (2SR [4]), which measures the temporal hue change from the spatial subspace rotation of skin-pixels as the pulse; and (iii) model-based approaches (CHROM [5], PBV [6] and POS [7]), which exploit characteristic properties of skin reflections (e.g., typical color absorption variations due to blood volume changes in living skin-tissues) and different assumptions on the distortions in the color channels to design a projection function from which the pulse-signal is extracted. A thorough review of these algorithms can be found in [7]. Among them, the model-based approaches demonstrate superior robustness in dealing with practical challenges such as the skin-tone variations, body-motions and illumination conditions.…”

“…In contrast, BSS-based approaches that do not use such priors need more data to get the high-quality statistics for solving the source de-mixing problem. In general, there are two major biometric signatures being used by current rPPG algorithms: (i) characteristic color absorptions (i.e., pulse-induced color variation directions in a multi-spectrum camera), which has been exploited by model-based rPPG [5][6][7] to differentiate between the pulse-induced color changes and noise-induced color changes; (ii) characteristic pulse-frequency range, which has been used in (almost) all rPPG algorithms to eliminate the frequency noise outside the assumed frequency-band (e.g., [40,240] beat per minute (bpm)), using a Band-Pass Filter (BPF).…”

Attacks on Heartbeat-Based Security Using Remote Photoplethysmography

“…A widely applicable and reliable remote attack would overcome these issues to maximize the probability of breaching security. While several such rPPG algorithms have recently been proposed [32], the evaluation of these algorithms rarely reports on how accurately individual heartbeats can be detected, making it difficult to directly assess the feasibility of a remote attacks from existing studies. To this end, we choose to implement and evaluate three state-of-theart rPPG algorithms 1 which -based on a recently published survey [32] -are expected to have a high detection accuracy:…”