Visual-only discrimination between native and non-native speech

Georgakis, Christos; Petridis, Stavros; Pantić, Maja

doi:10.1109/icassp.2014.6854519

Cited by 7 publications

(6 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a matter of fact, the latter have been shown to outperform uni-modal frameworks in various related tasks such as continuous interest prediction [40,16], detection of behavioral mimicry [41], and dimensional and continuous affect prediction [39], to mention but a few. Notably, other challenging problems such as accent classification [42,43,44] and pain intensity estimation [45] have been addressed based exclusively on visual features.…”

Section: Featuresmentioning

confidence: 99%

The Conflict Escalation Resolution (CONFER) Database

Georgakis

Panagakis

Zafeiriou

et al. 2017

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

Conflict is usually defined as a high level of disagreement taking place when individuals act on incompatible goals, interests, or intentions. Research in human sciences has recognized conflict as one of the main dimensions along which an interaction is perceived and assessed. Hence, automatic estimation of conflict intensity in naturalistic conversations would be a valuable tool for the advancement of human-centered computing and the deployment of novel applications for social skills enhancement including conflict management and negotiation. However, machine analysis of conflict is still limited to just a few works, partially due to an overall lack of suitable annotated data, while it has been mostly approached as a conflict or (dis)agreement detection problem based on audio features only. In this work, we aim to overcome the aforementioned limitations by a) presenting the Conflict Escalation Resolution (CONFER) Database, a set of excerpts from audio-visual recordings of televised political debates where conflicts naturally arise, and b) reporting baseline experiments on audio-visual conflict intensity estimation. The database contains approximately 142 minutes of recordings in Greek language, split over 120 non-overlapping episodes of naturalistic conversations that involve two or three interactants. Subject-and session-independent experiments are conducted on continuous-time (frame-by-frame) estimation of real-valued conflict intensity, as opposed to binary conflict/non-conflict clas- * Corresponding author. E-mail address: christos.georgakis@imperial.ac.uk Preprint submitted to Image and Vision Computing December 20, 2016 A C C E P T E D M A N U S C R I P T ACCEPTED MANUSCRIPTsification. For the problem at hand, the efficiency of various audio and visual features and fusion of them as well as various regression frameworks is examined. Experimental results suggest that there is much room for improvement in the design and development of automated multi-modal approaches to continuous conflict analysis. The CONFER Database is publicly available for non-commercial use at http://ibug.doc.ic.ac.uk/resources/confer/.

show abstract

Section: Featuresmentioning

confidence: 99%

The Conflict Escalation Resolution (CONFER) Database

Georgakis

Panagakis

Zafeiriou

et al. 2017

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…The lip contour extraction of curves is based on two formulas (3) and (4). Formula (3) presents the lip curve y 1 while formula (4) presents the lip curve y 2 .…”

Section: Face and Lip Detection Processmentioning

confidence: 99%

“…Here, three techniques have been employed for visual feature detection and description. Such techniques are SURF [3] (speeded up robust features), HoG [4] (histogram of oriented gradient) and Haar [5] for extracting lip contour features from visual source. SURF technique works by transforming the source image into coordinates using the integral image algorithm on ECG signals [6], which rapidly calculates summations of pixels over image sub-regions in constant time.…”

Section: Introductionmentioning

confidence: 99%

A Silent Password Recognition Framework Based on Lip Analysis

2020

View full text Add to dashboard Cite

Securing passwords either written or spoken is considered one of the challenging authentication issues faced by individuals and organizations. Written passwords could be easily stolen by look over, or manbehind, while spoken passwords could be recorded and replayed by attackers. A proposed silent password which is based on a dual security model for lip movement analysis will be a promoting solution to these attacks. The goal of the current research is to propose a hybrid voting framework for silent passwords recognition using lip movement analysis. The proposed framework is built for Arabic language by extracting figures from predefined Arabic lexicon. The predefined lexicon mainly contains the Arabic figures from zero to nine with different shapes. The framework takes a video or sequence of images as an input, and outputs the corresponding silent password for the frames extracted from the input video. In this paper, three techniques will be employed to extract effective visual features from mouth-lip movement. Such techniques are SURF, HoG and Haar feature extractor. The resultant features in each technique are fed separately into a classification model, namely, the hidden Markov model (HMM). The HMM identifies corresponding Arabic figure from a predefined lexicon based on input features. The final classification models that are produced from the three techniques have been grouped in a voting scheme to produce the final classification result. The proposed model will be tested on handcrafted data set of lip movement, and it has shown a promising result with improved accuracy of Arabic figures recognition. INDEX TERMS Hidden Markov model, lip analysis, silent password, voting scheme.

show abstract

“…In this paper, three techniques have been employed for visual feature detection and description. Such techniques are SURF [3](speeded up robust features), HoG [4] (histogram of oriented gradient) and Haar [5]for extracting lip contour features from visual source. These techniques have been chosen based on their remarkable efficacy and reported accuracy in different domains, for example in [6], [7] and [8].…”

Section: Introductionmentioning

confidence: 99%

An Arabic Figures Recognition Model Based on Automatic Learning of Lip Movement

Reda

Nasr

Ezz

et al. 2017

Journal of Al-Azhar University Engineering Sector

View full text Add to dashboard Cite

The need for an automatic speech to text conversion is continuously increasing, especially for people with special needs. Thus, automatic speech recognition techniques have been proposed to tackle such needs. The automatic recognition allows computers to identify words from lip movement, regardless of the visual source. It is known that visual speech recognition techniques improve the accuracy of word identification and shield the recognition system against acoustic noise. In this paper, we propose a hybrid voting model for automatic Arabic figures recognition based on visual perception of mouth-lip movement. The proposed model has been built for Arabic language, such that, it is able to extract Arabic figures from predefined Arabic lexicon. The predefined lexicon mainly contains the Arabic figures from zero to nine with different shapes. The model takes a video (or sequence of images) as an input, and outputs the corresponding Arabic figure for the frames extracted from the input video. Here, three techniques have been employed to extract effective visual features from mouth-lip movement. Such techniques are SURF (speeded up robust features), HoG (histogram of oriented gradient) and Haar feature extractor. The resultant features in each technique are fed separately into a classification model, namely, the hidden Markov model (HMM). The HMM identifies corresponding Arabic figure from a predefined lexicon based on input features. The final classification models that are produced from the three techniques have been grouped in a voting scheme to produce the final classification result (i.e. classification by voting). The proposed model in this paper has been tested on handcrafted data set of lip movement, and it has shown a promising result with improved accuracy of Arabic figures recognition.

show abstract

Visual-only discrimination between native and non-native speech

Cited by 7 publications

References 17 publications

The Conflict Escalation Resolution (CONFER) Database

The Conflict Escalation Resolution (CONFER) Database

A Silent Password Recognition Framework Based on Lip Analysis

An Arabic Figures Recognition Model Based on Automatic Learning of Lip Movement

Contact Info

Product

Resources

About