Visage: A Face Interpretation Engine for Smartphone Applications

Abstract

Abstract. Smartphones represent powerful mobile computing devices enabling a wide variety of new applications and opportunities for human interaction, sensing and communications. Because smartphones come with front-facing cameras, it is now possible for users to interact and drive applications based on their facial responses to enable participatory and opportunistic face-aware applications. This paper presents the design, implementation and evaluation of a robust, real-time face interpretation engine for smartphones, called Visage, that enables a new class of face-aware applications for smartphones. Visage fuses data streams from the phone's front-facing camera and built-in motion sensors to infer, in an energy-e cient manner, the user's 3D head poses (i.e., the pitch, roll and yaw of user's heads with respect to the phone) and facial expressions (e.g., happy, sad, angry, etc.). Visage supports a set of novel sensing, tracking, and machine learning algorithms on the phone, which are specifically designed to deal with challenges presented by user mobility, varying phone contexts, and resource limitations. Results demonstrate that Visage is e↵ective in di↵erent real-world scenarios. Furthermore, we developed two distinct proof-of-concept applications, Streetview+ and Mood Profiler driven by Visage.

Face Technologies on Mobile Devices

“…To cite another example, Visage system [27] tracks the face, detects the facial features, and uses the facial features to recognize user emotion, e.g., sad or happy. Two proof-of-concept applications are built on top of Visage to demonstrate its applicability.…”

Review on Stimuli Presentation for Affect Analysis Based on EEG

“…Visage: This is a prototype user interface environment for exploring and analyzing information. It represents an approach to coordinate multiple visualizations, analysis and presentation tools in data-intensive do-mains [119].…”

Toward a non-intrusive, physio- behavioral biometric for smartphones

“…The accelerometer and gyroscope sensors were used in [15] to infer and compensate tilted images for face detection. We propose using the sensors that follow: § Proximity sensors measure the proximity of an object relative to the device screen They can help infer when a user is placing a call (images will be discarded) or when the user is tapping the screen (face might be occluded by a finger).…”