Decomposing Multispectral Face Images into Diffuse and Specular Shading and Biophysical Parameters

Alotaibi, Sarah; Smith, William A. P.

doi:10.1109/icip.2019.8803369

Cited by 4 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The concept of the decomposition structure is based on the reference study in Ref. 18. The referenced structure has been modified and extended to include a skin segmentation model, patch divider and combiner, and virtual ColorChecker in Fig.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution

Jung¹,

Kim²,

Lee³

et al. 2023

J. Biomed. Opt.

View full text Add to dashboard Cite

Significance: Melanin and hemoglobin have been measured as important diagnostic indicators of facial skin conditions for aesthetic and diagnostic purposes. Commercial clinical equipment provides reliable analysis results, but it has several drawbacks: exclusive to the acquisition system, expensive, and computationally intensive.Aim: We propose an approach to alleviate those drawbacks using a deep learning model trained to solve the forward problem of light-tissue interactions. The model is structurally extensible for various light sources and cameras and maintains the input image resolution for medical applications.Approach: A facial image is divided into multiple patches and decomposed into melanin, hemoglobin, shading, and specular maps. The outputs are reconstructed into a facial image by solving the forward problem over skin areas. As learning progresses, the difference between the reconstructed image and input image is reduced, resulting in the melanin and hemoglobin maps becoming closer to their distribution of the input image. Results:The proposed approach was evaluated on 30 subjects using the professional clinical system, VISIA VAESTRO. The correlation coefficients for melanin and hemoglobin were found to be 0.932 and 0.857, respectively. Additionally, this approach was applied to simulated images with varying amounts of melanin and hemoglobin. Conclusion:The proposed approach showed high correlation with the clinical system for analyzing melanin and hemoglobin distribution, indicating its potential for accurate diagnosis. Further calibration studies using clinical equipment can enhance its diagnostic ability. The structurally extensible model makes it a promising tool for various image acquisition conditions.

show abstract

Section: Methodsmentioning

confidence: 99%

“…In this study, the skin reflectance calculated from Refs. 18, 34, and 44 is used based on the Kubelka-Munk theory. The volume fraction of the melanin is limited to a range of 1.3% to 43%, and the hemoglobin is in the range of 2% to 7% 45 …”

Section: Methodsmentioning

confidence: 99%

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution

Jung¹,

Kim²,

Lee³

et al. 2023

J. Biomed. Opt.

View full text Add to dashboard Cite

show abstract

“…The skin reflectance,

R( \lambda )

, is characterized by the path of light from illumination to reflection from the dermis layer: the fraction of light transmitted through the epidermis layer

{T_{epidrmis}}( \lambda )

, the fraction of light reflected from the dermis layer

{R_{dermis}}( \lambda )

, followed by an additional multiplication by

{T_{epidermis}}( \lambda )

. Using this approach with the conditions of volume fractions for melanin and hemoglobin range from 1.3% to 43% and from 2% to 7%, respectively, 31,32 we have applied the reflectance values used in previous studies 23,33 to our research.…”

Section: Methodsmentioning

confidence: 99%

“…In our previous research, 22 we employed the skin decomposition principle from the field of computer graphics 23 to train a skin analysis model for dermatological research. From general skin images, this model produces four outputs: melanin, hemoglobin, shading, and specular maps, and yields results comparable to those of clinical diagnostic equipment.…”

Section: Methodsmentioning

confidence: 99%

Generation of skin tone and pigmented region‐modified images using a pigment discrimination model trained with an optical approach

Jung,

Kim,

Lee

et al. 2023

Skin Research and Technology

View full text Add to dashboard Cite

BackgroundSkin tone and pigmented regions, associated with melanin and hemoglobin, are critical indicators of skin condition. While most prior research focuses on pigment analysis, the capability to simulate diverse pigmentation conditions could greatly broaden the range of applications. However, current methodologies have limitations in terms of numerical control and versatility.MethodsWe introduce a hybrid technique that integrates optical methods with deep learning to produce skin tone and pigmented region‐modified images with numerical control. The pigment discrimination model produces melanin, hemoglobin, and shading maps from skin images. The outputs are reconstructed into skin images using a forward problem‐solving approach, with model training aimed at minimizing the discrepancy between the reconstructed and input images. By adjusting the melanin and hemoglobin maps, we create pigment‐modified images, allowing precise control over changes in melanin and hemoglobin levels. Changes in pigmentation are quantified using the individual typology angle (ITA) for skin tone and melanin and erythema indices for pigmented regions, validating the intended modifications.ResultsThe pigment discrimination model achieved correlation coefficients with clinical equipment of 0.915 for melanin and 0.931 for hemoglobin. The alterations in the melanin and hemoglobin maps exhibit a proportional correlation with the ITA and pigment indices in both quantitative and qualitative assessments. Additionally, regions overlaying melanin and hemoglobin are demonstrated to verify independent adjustments.ConclusionThe proposed method offers an approach to generate modified images of skin tone and pigmented regions. Potential applications include visualizing alterations for clinical assessments, simulating the effects of skincare products, and generating datasets for deep learning.

show abstract

“…According to the Lambertian reflectance model [44], the reflectance light intensity I(p) of pixel p can be written as:…”

Section: B Depth-image Preprocessing and Multi-modality Fusionmentioning

confidence: 99%

Data-Fusion-Based Two-Stage Cascade Framework for Multimodality Face Anti-Spoofing

Liu

Wei²,

Lei

et al. 2022

IEEE Trans. Cogn. Dev. Syst.

View full text Add to dashboard Cite

Existing face anti-spoofing models using deep learning for multi-modality data suffer from low generalization in the case of using variety of presentation attacks such as 2D printing and high-precision 3D face masks. One of the main reasons is that the non-linearity of multi-spectral information used to preserve the intrinsic attributes between a real and a fake face are not well extracted. To address this issue, we propose a multi-modility data based two-stage cascade framework for face anti-spoofing. The proposed framework has two advantages. Firstly, we design a two-stage cascade architecture that can selectively fuse lowlevel and high-level features from different modalities to improve feature representation. Secondly, we use multi-modality data to construct a distance-free spectral on RGB and infrared (IR) to augment the non-linearity of data. The presented data fusion strategy is different from popular fusion approaches, since it can strengthen discrimination ability of network models on physical attribute features than identity structure features under certain constraints. In addition, a multi-scale patch based weighted finetuning strategy is designed to learn each specific local face region. Experimental results show that the proposed framework achieves better performance than other state-of-the-art methods on both benchmark datasets and self-established datasets, especially on multi-material masks spoofing.

show abstract

Decomposing Multispectral Face Images into Diffuse and Specular Shading and Biophysical Parameters

Cited by 4 publications

References 24 publications

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution

Deep learning-based optical approach for skin analysis of melanin and hemoglobin distribution

Generation of skin tone and pigmented region‐modified images using a pigment discrimination model trained with an optical approach

Data-Fusion-Based Two-Stage Cascade Framework for Multimodality Face Anti-Spoofing

Contact Info

Product

Resources

About