A new multi-purpose audio-visual UNMC-VIER database with multiple variabilities

Wong, Yee Wan; Ch’ng, Sue Inn; Seng, Kah Phooi; Ang, Li-Minn; Chin, Siew Wen; Chew, Wei Jen; Lim, King Hann

doi:10.1016/j.patrec.2011.06.011

Cited by 14 publications

(7 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…On the continuous speech datasets, it is interesting to note that most still use speaker-dependent tests [76,58,147,28]. We note that some are single speaker-dependent, others multi-speaker dependent, the crux of the point is that test speaker samples are included in the training data.…”

Section: Speaker Independencementioning

confidence: 99%

Decoding visemes: Improving machine lip-reading

Bear

Harvey

2016

2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

To undertake machine lip-reading, we try to recognise speech from a visual signal. Current work often uses viseme classification supported by language models with varying degrees of success. A few recent works suggest phoneme classification, in the right circumstances, can outperform viseme classification. In this work we present a novel two-pass method of training phoneme classifiers which uses previously trained visemes in the first pass. With our new training algorithm, we show classification performance which significantly improves on previous lip-reading results.

show abstract

Section: Speaker Independencementioning

confidence: 99%

Decoding visemes: Improving machine lip-reading

Bear

Harvey

2016

2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

View full text Add to dashboard Cite

show abstract

“…CMU AVPFV [190], IV2 [192], BL [196], and UNMC-VIER [197] all place a camera on the front and single side of the speaker, and collect the front and side views at the same time. The IBMSR [164] database placed three cameras on the front and both sides of the speaker and collects views from three angles at the same time, but unfortunately, the IBMSR database was not public.…”

Section: ) Multi View Databasesmentioning

confidence: 99%

A Survey of Research on Lipreading Technology

et al. 2020

View full text Add to dashboard Cite

“…This is made up of 11 females and 31 males, from 6 different countries and ages ranging from 19 to 59 years. Although FAVLIPS is a diverse dataset it is still considered small in comparison to [24][25][26]; however, it fills a need for testing a LBBA with real-world data. The content captured provides a wide range of real-world testing situations such as miming and lighting that could be expected if deployed on a mobile device.…”

Section: Favlipsmentioning

confidence: 99%

Understanding visual lip-based biometric authentication for mobile devices

Wright¹,

Stewart²

2020

EURASIP J. on Info. Security

View full text Add to dashboard Cite

This paper explores the suitability of lip-based authentication as a behavioural biometric for mobile devices. Lip-based biometric authentication is the process of verifying an individual based on visual information taken from the lips while speaking. It is particularly suited to mobile devices because it contains unique information; its potential for liveness over existing popular biometrics such as face and fingerprint and lip movements can be captured using a device's front-facing camera, requiring no dedicated hardware. Despite its potential, research and progress into lip-based biometric authentication has been significantly slower than other biometrics such as face, fingerprints, or iris. This paper investigates a state-of-the-art approach using a deep Siamese network, trained with the triplet loss for one-shot lip-based biometric authentication with real-world challenges. The proposed system, LipAuth, is rigourously examined with real-world data and challenges that could be expected on lip-based solution deployed on a mobile device. The work in this paper shows for the first time how a lip-based authentication system performs beyond a closed-set protocol, benchmarking a new open-set protocol with an equal error rates of 1.65% on the XM2VTS dataset. New datasets, qFace and FAVLIPS, were collected for the work in this paper, which push the field forward by enabling systematic testing of the content and quantities of data needed for lip-based biometric authentication and highlight problematic areas for future work. The FAVLIPS dataset was designed to mimic some of the hardest challenges that could be expected in a deployment scenario and include varied spoken content, miming and a wide range of challenging lighting conditions. The datasets captured for this work are available to other university research groups on request.

show abstract

A new multi-purpose audio-visual UNMC-VIER database with multiple variabilities

Cited by 14 publications

References 18 publications

Decoding visemes: Improving machine lip-reading

Decoding visemes: Improving machine lip-reading

A Survey of Research on Lipreading Technology

Understanding visual lip-based biometric authentication for mobile devices

Contact Info

Product

Resources

About