No abstract

No abstract

No abstract

Abstract. A robust method for locating and tracking lips in gray-level image sequences is described. The method learns patterns of shape variability from a training set which constrains the model during image search to only deform in ways similar to the training examples. Image search is guided by a learned gray-level model which is used to describe the large appearance variability of lips. Such v ariability might be due to di erent individuals, illumination, mouth opening, specularity or visibility of teeth and tongue. Visual speech features are recovered from the tracking results and represent both, shape and intensity information. We describe a speechreading (lip-reading) system, where the extracted features are modeled by Gaussian distributions and their temporal dependencies by Hidden Markov Models. Experimental results are presented for locating lips, tracking lips and speechreading. The database used consists of a broad variety o f s p e a k ers and was recorded in a natural environment with no special lighting or lip markers used. For a speaker independent digit recognition task using visual information only, the system achieved an accuracy about equivalent to that of untrained humans. IDIAP{RR 97-12

No abstract