Lip Biometrics for Digit Recognition

Abstract

Abstract. This paper presents a speaker-independent audio-visual digit recognition system that utilizes speech and visual lip signals. The extracted visual features are based on line-motion estimation obtained from video sequences with low resolution (128 ×128 pixels) to increase the robustness of audio recognition. The core experiments investigate lip motion biometrics as stand-alone as well as merged modality in speech recognition system. It uses Support Vector Machines, showing favourable experimental results with digit recognition featuring 83% to 100% on the XM2VTS database depending on the amount of available visual information.

Verifying liveness by multiple experts in face biometrics

“…Presumably, we cannot rely on eye-blinking or utterances, even on command, assuming an attacker that uses a photographic mask (therefore these measures are written in parenthesis). Are they sufficient against spoofing in biometrics [9,3]? We have studied this issue in our experiments, which we will report further below.…”

“…Some approaches involved the speech modality ("talking face") [1,4], because the fused audio-visual features are easier to classify into live/non-live. Within this paradigm, that targets on attacks both via photographs and videos, [3] is of interest on its own because the latter showed the possibility of recognizing utterances (digits 0-9) from lip motion only. These multimodal biometrics, although highly useful for both verification and liveness detection purposes, rely on very controlled conditions and are complementary to our study.…”

“…The ingredients in our liveness detection scheme are face (part) detection and motion estimation/tracking. In particular, motion estimation is an essential component of liveness detection since it assists in many countermeasures, for example, of type II and III as shown in [3] and [7], respectively. As a novelty we are additionally suggesting it for eye-blinking detection here, and exploit it also for face tracking.…”

Lip-motion events analysis and lip segmentation using optical flow

“…We will address these issues by introducing an optical flow algorithm well suited for the task, and provide the methods of producing robust features efficiently. The importance of robust features and to deal with the case of linear motions for lips, is underlined in the work of [8]. A further problem in the case of lip motions lies with prescence of multiple motions, or motion boundaries, a topic we do not deal with in this work.…”