Lip as biometric and beyond: a survey

Chowdhury, Debbrota Paul; Kumari, Ritu; Bakshi, Sambit; Sahoo, Manmath Narayan; Das, Abhijit

doi:10.1007/s11042-021-11613-5

Cited by 15 publications

(20 citation statements)

References 150 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The numerator is the sum of the exponential function of prediction values in each class (z c ). The total number of classes is represented as C. The cross-entropy loss is shown in equation (1).…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A lightweight deep learning approach to mouth segmentation in color images

Chotikkakamthorn

Ritthipravat²,

Kusakunniran³

et al. 2022

ACI

View full text Add to dashboard Cite

PurposeMouth segmentation is one of the challenging tasks of development in lip reading applications due to illumination, low chromatic contrast and complex mouth appearance. Recently, deep learning methods effectively solved mouth segmentation problems with state-of-the-art performances. This study presents a modified Mobile DeepLabV3 based technique with a comprehensive evaluation based on mouth datasets.Design/methodology/approachThis paper presents a novel approach to mouth segmentation by Mobile DeepLabV3 technique with integrating decode and auxiliary heads. Extensive data augmentation, online hard example mining (OHEM) and transfer learning have been applied. CelebAMask-HQ and the mouth dataset from 15 healthy subjects in the department of rehabilitation medicine, Ramathibodi hospital, are used in validation for mouth segmentation performance.FindingsExtensive data augmentation, OHEM and transfer learning had been performed in this study. This technique achieved better performance on CelebAMask-HQ than existing segmentation techniques with a mean Jaccard similarity coefficient (JSC), mean classification accuracy and mean Dice similarity coefficient (DSC) of 0.8640, 93.34% and 0.9267, respectively. This technique also achieved better performance on the mouth dataset with a mean JSC, mean classification accuracy and mean DSC of 0.8834, 94.87% and 0.9367, respectively. The proposed technique achieved inference time usage per image of 48.12 ms.Originality/valueThe modified Mobile DeepLabV3 technique was developed with extensive data augmentation, OHEM and transfer learning. This technique gained better mouth segmentation performance than existing techniques. This makes it suitable for implementation in further lip-reading applications.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Mouth segmentation is an important process in lip reading that can be applied in several applications, such as video conferencing, lip-synching, visual face recognition, speech recognition and medical disease detection [1][2][3][4]. The accuracy of each application depends on segmentation performances.…”

Section: Introductionmentioning

confidence: 99%

A lightweight deep learning approach to mouth segmentation in color images

Chotikkakamthorn

Ritthipravat²,

Kusakunniran³

et al. 2022

ACI

View full text Add to dashboard Cite

show abstract

“…One study [ 485 ] indicated their ability by visual comparison to distinguish marks and prints from 102 subjects (52 males and 50 females). The research in the future may benefit more from machine learning techniques developed for in biometric applications [ 486 ]. However, at the moment, efforts are more devoted to lip images as a visible external attribute than on marks or prints left on surfaces.…”

Section: Other Body Marks Case Reports and Miscellaneousmentioning

confidence: 99%

Interpol review of fingermarks and other body impressions (2019 – 2022)

Bécue

Champod

2023

Forensic Science International: Synergy

View full text Add to dashboard Cite

“…Some of the futuristic applications of VSR include biometrics [6]. This technology is called lip biometrics or visual speech biometrics [7], [8]. Lip biometrics captures and analyzes a video of a person speaking to extract the lip movements and facial features.…”

Section: Futuristic Applications Of Speaker Independent Vsrmentioning

confidence: 99%

Deep Learning-Based Holistic Speaker Independent Visual Speech Recognition

Nemani¹,

Krishna²,

Ramisetty³

et al. 2023

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

Speaker-independent visual speech recognition (VSR) is a complex task that involves identifying spoken words or phrases from video recordings of a speaker's facial movements. Decoding the intricate visual dynamics of a speaker's mouth in a high-dimensional space is a significant challenge in this field. To address this challenge, researchers have employed advanced techniques that enable machines to recognize human speech through visual cues automatically. Over the years, there has been a considerable amount of research in the field of VSR involving different algorithms and datasets to evaluate system performance. These efforts have resulted in significant progress in developing effective VSR models, creating new opportunities for further research in this area. This survey provides a detailed examination of the progression of VSR over the past three decades, with a particular emphasis on the transition from speaker-dependent to speaker-independent systems. We also provide a comprehensive overview of the various datasets used in VSR research and the preprocessing techniques employed to achieve speaker independence. The survey covers the works published from 1990 to 2023, thoroughly analyzing each work and comparing them on various parameters. This survey provides an in-depth analysis of speaker-independent VSR systems evolution from 1990 to 2023. It outlines the development of VSR systems over time and highlights the need to develop end-to-end pipelines for speakerindependent VSR. The pictorial representation offers a clear and concise overview of the techniques used in speaker-independent VSR, thereby aiding in the comprehension and analysis of the various methodologies. The survey also highlights the strengths and limitations of each technique and provides insights into developing novel approaches for analyzing visual speech cues. Overall, This comprehensive review provides insights into the current state-of-the-art speaker-independent VSR and highlights potential areas for future research.

show abstract

Lip as biometric and beyond: a survey

Cited by 15 publications

References 150 publications

A lightweight deep learning approach to mouth segmentation in color images

A lightweight deep learning approach to mouth segmentation in color images

Interpol review of fingermarks and other body impressions (2019 – 2022)

Deep Learning-Based Holistic Speaker Independent Visual Speech Recognition

Contact Info

Product

Resources

About