Automatic Lip Reading in the Dutch Language Using Active Appearance Models on High Speed Recordings

Chitu, Alin; Driel, Karin; Rothkrantz, Léon J. M.

doi:10.1007/978-3-642-15760-8_33

Cited by 18 publications

(12 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We used a fitting procedure for a 3D AAM, based on Kernel methods for regression. The statistics computed on data generated with our 3D AAM implementation show that the kernel methods give better results compared to the linear regression models [37,42,43,54,69,76,79,89,91,99].…”

Section: Analysis Of Facial Expressions and Nonverbal Behaviour At Thmentioning

confidence: 99%

Surveillance Angels : Tutorial

Rothkrantz¹

2014

NNW

View full text Add to dashboard Cite

The use of sensor networks has been proposed for military surveillance and environmental monitoring applications. Those systems are composed of a heterogeneous set of sensors to observe the environment. In centralised systems the observed data will be conveyed to the control room to process the data. Human operators are supposed to give a semantic interpretation of the observed data. They are searching for suspicious or unwanted behaviour. The increase of surveillance sensors in the military domain requires a huge amount of human operators which is far beyond available resources. Automated systems are needed to give a context sensitive semantic interpretation of the observed kinematic data. As a proof of concept two automatic surveillance projects will be discussed in this paper. The first project is about a centralised system based on the AIS-Automated Identification System which will be used to monitor ship movements automatically. The second project is about a decentralised system composed of a network of cameras installed at a military area. There is a need for a surveillance system along the coast of Europe. There is an increase of illegal drugs transport from the open sea, intrusion of boat refuges, illegal fishing, pollution of the sea by illegal chemical and oil pollution by ships. An automated sensor system is needed to detect illegal intruders and suspicious ship movements. Vessels fitted with AIS transceivers and transponders can be tracked by AIS base stations located along coast lines or, when out of range of terrestrial networks, through a growing number of satellites that are fitted with special AIS receivers. AIS data include a unique identifier of a vessel and kinematic data such as its position, course and speed. The proposed system enables identification, and tracking of vessels and to detect unwanted or illegal behaviour of ship movements. If ships violate traffic rules, enter forbidden areas or approach a critical infrastructure an alert will be generated automatically in the control room. Human operators start an emergency procedure. The second project is about a network of cameras installed at a military area. The area is monitored by multiple cameras with non-overlapping field of views monitored by human operators. We developed an automated surveillance system. At the entrance gate the identity of visitors will be checked by a face recognition system. In case of intruders, unwanted behaviour, trouble makers the emotional state of the visitor will be assessed by an analysis of facial expressions using the Active Appearance model. If unwanted behaviour is detected an alert is send the control room.

show abstract

Section: Analysis Of Facial Expressions and Nonverbal Behaviour At Thmentioning

confidence: 99%

Surveillance Angels : Tutorial

Rothkrantz¹

2014

NNW

View full text Add to dashboard Cite

show abstract

“…IV2 [192] database was a sentence level database based on French, with 300 people participating in the recording, each speaking 15 French sentences. There are also Czech databases UWB-05-HSAVC [187] and UWB-07-ICAV [193], NDUTAVSC [168] database for German, HAVRUS [199] database for Russian, BL [196] database for French, and VLRF [202] database for Spanish.…”

Section: ) Word Phrase and Sentence Recognitionmentioning

confidence: 99%

A Survey of Research on Lipreading Technology

et al. 2020

View full text Add to dashboard Cite

“…Other datasets already described in Section 2.1 for digit recognition also contain specific sessions with sentences: AV@CAR provides 250 phoneticallybalanced sentences, AVICAR sentences with more than 1,300 different words, and AVOZES three different sentences designed to contain almost all phonemes and visemes of Australian English. the NDUTAVSC database [72], with 66 German speakers, the AV@CAR database [63], in Spanish (already described above) and the VLRF database [37], also in Spanish, providing 1,507 utterances from 24 speakers. In Table 2 we show examples of sentences of some of 355 these AV-databases.…”

Section: Word and Sentence Recognitionmentioning

confidence: 99%

Survey on automatic lip-reading in the era of deep learning

Fernandez-Lopez

Sukno

2018

Image and Vision Computing

View full text Add to dashboard Cite

In the last few years, there has been an increasing interest in developing systems for Automatic LipReading (ALR). Similarly to other computer vision applications, methods based on Deep Learning (DL) have become very popular and have permitted to substantially push forward the achievable performance. In this survey, we review ALR research during the last decade, highlighting the progression from approaches previous to DL (which we refer to as traditional) toward end-to-end DL architectures. We provide a comprehensive list of the audiovisual databases available for lipreading, describing what tasks they can be used for, their popularity and their most important characteristics, such as the number of speakers, vocabulary size, recording settings and total duration. In correspondence with the shift toward DL, we show that there is a clear tendency toward large-scale datasets targeting realistic application settings and large numbers of samples per class. On the other hand, we summarize, discuss and compare the different ALR systems proposed in the last decade, separately considering traditional and DL approaches. We address a quantitative analysis of the different systems by organizing them in terms of the task that they target (e.g. recognition of letters or digits and words or sentences) and comparing their reported performance in the most commonly used datasets. As a result, we find that DL architectures perform similarly to traditional ones for simpler tasks but report significant improvements in more complex tasks, such as word or sentence recognition, with up to 40% improvement in word recognition rates. Hence, we provide a detailed description of the available ALR systems based on end-to-end DL architectures and identify a tendency to focus on the modeling of temporal context as the key to advance the field. Such modeling is dominated by recurrent neural networks due to their ability to retain context at multiple scales (e.g. short-and longterm information). In this sense, current efforts tend toward techniques that allow a more comprehensive modeling and interpretability of the retained context.

show abstract

Automatic Lip Reading in the Dutch Language Using Active Appearance Models on High Speed Recordings

Cited by 18 publications

References 9 publications

Surveillance Angels : Tutorial

Surveillance Angels : Tutorial

A Survey of Research on Lipreading Technology

Survey on automatic lip-reading in the era of deep learning

Contact Info

Product

Resources

About