Face Swapping: Realistic Image Synthesis Based on Facial Landmarks Alignment

Abstract

We propose an image-based face swapping algorithm, which can be used to replace the face in the reference image with the same facial shape and features as the input face. First, a face alignment is made based on a group of detected facial landmarks, so that the aligned input face and the reference face are consistent in size and posture. Secondly, an image warping algorithm based on triangulation is presented to adjust the reference face and its background according to the aligned input faces. In order to achieve more accurate face swapping, a face parsing algorithm is introduced to realize the accurate detection of the face-ROIs, and then the face-ROI in the reference image is replaced with the input face-ROI. Finally, a Poisson image editing algorithm is adopted to realize the boundary processing and color correction between the replacement region and the original background, and then the final face swapping result is obtained. In the experiments, we compare our method with other face swapping algorithms and make a qualitative and quantitative analysis to evaluate the reality and the fidelity of the replaced face. The analysis results show that our method has some advantages in the overall performance of swapping effect.

Adjusting eye aspect ratio for strong eye blink detection based on facial landmarks

“…Face tracking becomes strong for rigid facial deformation and not stiff due to head movements and facial expressions. Furthermore, facial landmarks were successfully applied to face alignment, head pose estimation, face swapping ( Chen et al, 2019 ), and blink detection ( Cao et al, 2021 ).…”

Peer Review #1 of "Adjusting eye aspect ratio for strong eye blink detection based on facial landmarks (v0.1)"

“…Face tracking becomes strong for rigid facial deformation and not stiff due to head movements and facial expressions. Furthermore, facial landmarks were successfully applied to face alignment, head pose estimation, face swapping (D. Chen et al, 2019), and blink detection (Cao et al, 2021).…”

Swapped face detection using deep learning and subjective assessment

“…Mahajan et al [10] present an algorithm that automatically chooses faces that are facing the front and then replaces them with stock faces in a similar fashion as Bitouk et al [9]. Chen et al [11] suggested an algorithm that can be used in the replacement of faces in referenced images that have common features and shape as the input face. A triangulation-based algorithm is used in warping the image by adjusting the reference face and its accompanying background to the input face.…”