Motion robust remote photoplethysmography in CIELab color space

Yang, Yuting; Liu, Chenbin; Yu, Hui Jing; Shao, Dangdang; Tsow, Francis; Tao, Nongjian

doi:10.1117/1.jbo.21.11.117001

Cited by 41 publications

(25 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…principal component analysis (PCA) (Lewandowska et al 2011) and independent component analysis (ICA) (Poh et al 2011, Tsouri et al 2012) to unmix the RGB-signals obtained by a camera into uncorrelated or independent signal sources and select the most periodic one as the pulse; (ii) Color-space driven methods, which measure the pulse in different standard color-spaces (i.e. HUE color-space (Tsouri and Li 2015) and lab color-space (Yang 2016)), i.e. some optical disturbances (e.g.…”

Section: Introductionmentioning

confidence: 99%

Robust heart rate from fitness videos

Wang

Brinker²,

Stuijk

et al. 2017

Physiol. Meas.

View full text Add to dashboard Cite

Remote photoplethysmography (rPPG) enables contactless heart-rate monitoring using a regular video camera. Objective: This paper aims to improve the rPPG technology targeting continuous heart-rate measurement during fitness exercises. The fundamental limitation of the existing (multi-wavelength) rPPG methods is that they can suppress at most n − 1 independent distortions by linearly combining n wavelength color channels. Their performance are highly restricted when more than n − 1 independent distortions appear in a measurement, as typically occurs in fitness applications with vigorous body motions. Approach: To mitigate this limitation, we propose an effective yet very simple method that algorithmically extends the number of possibly suppressed distortions without using more wavelengths. Our core idea is to increase the degrees-of-freedom of noise reduction by decomposing the n wavelength camera-signals into multiple orthogonal frequency bands and extracting the pulse-signal per band-basis. This processing, namely Sub-band rPPG (SB), can suppress different distortion-frequencies using independent combinations of color channels. Main results: A challenging fitness benchmark dataset is created, including 25 videos recorded from 7 healthy adult subjects (ages from 25 to 40 yrs; six male and one female) running on a treadmill in an indoor environment. Various practical challenges are simulated in the recordings, such as different skin-tones, light sources, illumination intensities, and exercising modes. The basic form of SB is benchmarked against a state-of-the-art method (POS) on the fitness dataset. Using non-biased parameter settings, the average signal-tonoise-ratio (SNR) for POS varies in [−4.18, −2.07] dB, for SB varies in [−1.08, 4.77] dB. The ANOVA test shows that the improvement of SB over POS is statistically significant for almost all settings (p-value <0.05). Significance: The results suggest that the proposed SB method considerably increases the

show abstract

Section: Introductionmentioning

confidence: 99%

Robust heart rate from fitness videos

Wang

Brinker²,

Stuijk

et al. 2017

Physiol. Meas.

View full text Add to dashboard Cite

show abstract

“…Rodríguez and Castro [ 25 ] applied a simple intensity threshold to exclude darker areas like the eyebrows. Yang et al [ 9 ] built a roughness measure in sub-ROIs which was employed to select the smoothest regions. Bousefsaf et al [ 26 ] used the lightness component of the CIE L*u*v space to create five regional clusters of which the best were eventually combined.…”

Section: Discussionmentioning

confidence: 99%

“…In our setup, common face detection algorithms, as used in [ 6 , 7 , 9 ], eventually failed due to the limited visibility of required features. To detect suitable regions that potentially provide physiological information, we employed a skin classifier by Jones and Rehg [ 29 ] on the (first) RGB image.…”

Section: Methodsmentioning

confidence: 99%

“…For this reason, the selection of spatially homogeneous ROIs is essential to achieve pure cbPPG signals. Previous approaches, which in some way considered the ROI’s homogeneity, employed intensity thresholds, exploited regional means and standard deviations or clustered areas based on the lightness component [ 9 , 25 , 26 ]. For the eventual application, all those methods depend on an initial face detection.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Camera-based photoplethysmography in an intraoperative setting

et al. 2018

View full text Add to dashboard Cite

BackgroundCamera-based photoplethysmography (cbPPG) is a measurement technique which enables remote vital sign monitoring by using cameras. To obtain valid plethysmograms, proper regions of interest (ROIs) have to be selected in the video data. Most automated selection methods rely on specific spatial or temporal features limiting a broader application. In this work, we present a new method which overcomes those drawbacks and, therefore, allows cbPPG to be applied in an intraoperative environment.MethodsWe recorded 41 patients during surgery using an RGB and a near-infrared (NIR) camera. A Bayesian skin classifier was employed to detect suitable regions, and a level set segmentation approach to define and track ROIs based on spatial homogeneity.ResultsThe results show stable and homogeneously illuminated ROIs. We further evaluated their quality with regards to extracted cbPPG signals. The green channel provided the best results where heart rates could be correctly estimated in 95.6% of cases. The NIR channel yielded the highest contribution in compensating false estimations.ConclusionsThe proposed method proved that cbPPG is applicable in intraoperative environments. It can be easily transferred to other settings regardless of which body site is considered.Electronic supplementary materialThe online version of this article (10.1186/s12938-018-0467-7) contains supplementary material, which is available to authorized users.

show abstract

“…2) Color Space Selection: As discussed in [40], head movement effects much more on the intensity than the chromaticity of the image since the chromaticity reflects the intrinsic optical properties of hemoglobin in blood. Therefore, choosing a color space separating chromaticity from intensity is helpful to reduce the artifacts and thus benefit the training procedure.…”

Section: Key Components Analysismentioning

confidence: 99%

RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation

Niu

Shan

Han

et al. 2020

IEEE Trans. on Image Process.

226

159

View full text Add to dashboard Cite

Heart rate (HR) is an important physiological signal that reflects the physical and emotional status of a person. Traditional HR measurements usually rely on contact monitors, which may cause inconvenience and discomfort. Recently, some methods have been proposed for remote HR estimation from face videos; however, most of them focus on well-controlled scenarios, their generalization ability into less-constrained scenarios (e.g., with head movement, and bad illumination) are not known. At the same time, lacking large-scale HR databases has limited the use of deep models for remote HR estimation. In this paper, we propose an end-to-end RhythmNet for remote HR estimation from the face. In RyhthmNet, we use a spatial-temporal representation encoding the HR signals from multiple ROI volumes as its input. Then the spatial-temporal representations are fed into a convolutional network for HR estimation. We also take into account the relationship of adjacent HR measurements from a video sequence via Gated Recurrent Unit (GRU) and achieves efficient HR measurement. In addition, we build a large-scale multi-modal HR database (named as VIPL-HR 1 ), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database contains various variations such as head movements, illumination variations, and acquisition device changes, replicating a less-constrained scenario for HR estimation. The proposed approach outperforms the stateof-the-art methods on both the public-domain and our VIPL-HR databases.

show abstract

Motion robust remote photoplethysmography in CIELab color space

Cited by 41 publications

References 27 publications

Robust heart rate from fitness videos

Robust heart rate from fitness videos

Camera-based photoplethysmography in an intraoperative setting

RhythmNet: End-to-End Heart Rate Estimation From Face via Spatial-Temporal Representation

Contact Info

Product

Resources

About