What Will Your Future Child Look Like? Modeling and Synthesis of Hereditary Patterns of Facial Dynamics

Ertuğrul, Itır Önal; Dibeklioğlu, Hamdi

doi:10.1109/fg.2017.14

Cited by 14 publications

(13 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We extend our previous study [10] in many ways. Along with an extended literature, (1) we use intensity of facial action units (AUs) instead of facial landmark displacement for both expression matching and learning temporal dynamics, (2) we model the facial appearance in a holistic manner, rather than learning facial regions individually since a set of AUs can effectively describe the whole face during expression matching, (3) we extend our dataset including the UvA-NEMO Disgust Database [9] and generate disgust videos of children in addition to their smile videos, (4) we enhance the reliability of the kinship verification method used in our experiments, and (5) we perform analyses to evaluate the quality of synthesized expressions in terms of occurrence and intensity of AUs.…”

Section: Introductionmentioning

confidence: 54%

“…In order to synthesize videos of children from videos of the corresponding parents, we employ the kinship set [4] of the UvA-NEMO Smile [8] and Disgust [9] databases, which are the only available kinship video databases in the literature. In our previous study [10], we have evaluated our model solely on UvA-NEMO Smile database. In the current study, to show the generalizability of the proposed method for other facial expressions, we include an evaluation on UvA-NEMO Disgust database.…”

Section: Databasementioning

confidence: 99%

“…We compare the kinship verification accuracy values obtained using facial landmarks [10] and AU intensities during expression matching and synthesis of dynamics on smile database in Fig. 6.…”

Section: Combining Appearance and Dynamicsmentioning

confidence: 99%

See 2 more Smart Citations

Modeling and synthesis of kinship patterns of facial expressions

Ertuğrul

Jeni

Dibeklioğlu

2018

Image and Vision Computing

View full text Add to dashboard Cite

Analysis of kinship from facial images or videos is an important problem. Prior machine learning and computer vision studies approach kinship analysis as a verification or recognition task. In this paper, for the first time in the literature, we propose a kinship synthesis framework, which generates smile and disgust videos of (probable) children from the expression videos (smile and disgust) of parents. While the appearance of a child's expression is learned using a convolutional encoder-decoder network, another neural network models the dynamics of the corresponding expression. The expression video of the estimated child is synthesized by the combined use of appearance and dynamics models. In order to validate our results, we perform kinship verification experiments using videos of real parents and estimated children generated by our framework. The results show that generated videos of children achieve higher correct verification rates than those of real children. Our results also indicate that the use of generated videos together with the real ones in the training of kinship verification models, increases the accuracy, suggesting that such videos can be used as a synthetic dataset. Furthermore, we evaluate the expression similarity between input and output frames, and show that the proposed method can fairly retain the expression of input faces while transforming the facial identity.

show abstract

Section: Introductionmentioning

confidence: 54%

Section: Databasementioning

confidence: 99%

See 1 more Smart Citation

Modeling and synthesis of kinship patterns of facial expressions

Ertuğrul

Jeni

Dibeklioğlu

2018

Image and Vision Computing

View full text Add to dashboard Cite

show abstract

“…Similarly, autoencoder-based (AE) methods have the similar drawbacks for the solution. [10] aims to generate kinship faces by promoting facial dynamics (i.e., expression) along with visual appearances based on AE, thus it is able to transfer personal expressions to prospective children.…”

Section: Face Synthesismentioning

confidence: 99%

Kinshipgan: Synthesizing of Kinship Faces from Family Photos by Regularizing a Deep Face Network

Özkan

Ozkan

2018

2018 25th IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

In this paper, we propose a kinship generator network that can synthesize a possible child face by analyzing his/her parent's photo. For this purpose, we focus on to handle the scarcity of kinship datasets throughout the paper by proposing novel solutions in particular. To extract robust features, we integrate a pre-trained face model to the kinship face generator. Moreover, the generator network is regularized with an additional face dataset and adversarial loss to decrease the overfitting of the limited samples. Lastly, we adapt cycle-domain transformation to attain a more stable results. Experiments are conducted on Families in the Wild (FIW) dataset. The experimental results show that the contributions presented in the paper provide important performance improvements compared to the baseline architecture and our proposed method yields promising perceptual results.

show abstract

“…Out of the many recent works in automatic kinship recognition, only a few have attempted the kinship generation problem. Ertugrul et al [4] focused on generating the facial Corresponding author: Siyu Xia. (e-mail: xia081@gmail.com).…”

Section: Introductionmentioning

confidence: 99%

DNA-Net: Age and Gender Aware Kin Face Synthesizer

Gao¹,

Robinson

Zhu³

et al. 2021

2021 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Visual kinship verification aims to detect blood relatives in facial images. Its practical application have motivated many researchers to focus on the topic as of recent. In this paper, we focus on a new view of visual kinship technology: kin-based face generation. Specifically, we propose a two-stage kin-face generation model to predict the appearance of a child given a pair of parents. The first stage includes a deep generative adversarial auto-encoder conditioned on ages and genders to map between facial appearance and high-level features. The second stage is our proposed DNA-Net, which serves as a transformation between the deep and genetic features based on a random selection process to fuse genes of a parent pair to form the genes of a child. We demonstrate the effectiveness of the proposed method quantitatively and qualitatively. Experiments validate that the proposed model synthesizes convincing kin-faces using both subjective and objective standards.

show abstract

What Will Your Future Child Look Like? Modeling and Synthesis of Hereditary Patterns of Facial Dynamics

Cited by 14 publications

References 26 publications

Modeling and synthesis of kinship patterns of facial expressions

Modeling and synthesis of kinship patterns of facial expressions

Kinshipgan: Synthesizing of Kinship Faces from Family Photos by Regularizing a Deep Face Network

DNA-Net: Age and Gender Aware Kin Face Synthesizer

Contact Info

Product

Resources

About