Making 3D Eyeglasses Try-on practical

Abstract

In this paper, we propose a virtual 3D Eyeglasses Try-on (3DET) system, with efficient, realistic and real-time augmented performance. The 3DET system captures user's performance with the help of depth camera and renders the glasses properly on the video stream immediately after user's selection. The virtual eye-glasses follow with the motion (movement or rotation) of user's head simultaneously. The high efficiency of our 3DET system is achieved by simplifying the eyeglasses matching procedure, making use of the active appearance model (AAM) based face tracking algorithm. This is completely different from existing methods, which usually relies on eye detection. In addition, due to the exploiting of a generic 3D face model during tracking and displaying, the 3DET system can handle occlusion problem easily and render realistic glasses in videos effectively. Experimental results demonstrate that the proposed 3DET system is able to produce superior natural and smooth visual results with virtual glasses fitted on the users face at 30 fps with a high level of accuracy on common hardware.

MEGANE: Morphable Eyeglass and Avatar Network

“…Eyeglasses Modeling. Eyeglasses are one of the most commonly used accessories in our daily life, and virtual try-on has been extensively studied [22,23,32,48,58,77,80]. An image-based eyeglasses try-on is possible by composing a glasses image onto a face using Poisson blending [32].…”