Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation

Kaltwang, Sebastian; Todorovic, Sinisa; Pantić, Maja

doi:10.1109/tpami.2015.2501824

Cited by 46 publications

(46 citation statements)

References 50 publications

(117 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a matter of fact, the latter have been shown to outperform uni-modal frameworks in various related tasks such as continuous interest prediction [40,16], detection of behavioral mimicry [41], and dimensional and continuous affect prediction [39], to mention but a few. Notably, other challenging problems such as accent classification [42,43,44] and pain intensity estimation [45] have been addressed based exclusively on visual features.…”

Section: Featuresmentioning

confidence: 99%

“…fective computing tasks such as face recognition [52] and pain intensity estimation [45] have relied on appearance features locally extracted from a pre-defined grid of rectangular regions in face images registered in frontal pose. However, this technique is not suitable for databases including images that portray faces with large head pose angles, as is the case with the CON-FER Database, since the 2D registration process unavoidably induces pixel artefacts and texture discontinuities.…”

Section: Accepted M Manuscriptmentioning

confidence: 99%

“…The Intra-class Correlation Coefficient (ICC) [72], initially proposed as a metric for rater reliability in behavioral measurements, has been recently applied in providing a measure of 'consistency' or 'agreement' between ground truth annotations of behavioral or affective attributes provided by humans and corresponding predictions yielded by automated approaches (e.g., [45,73]). It typically expresses the fraction of the total variance across all ratings and subjects (including random error in the 'judgements') 'explained' by the component of variance due to the targets alone [72].…”

Section: Evaluation Metricsmentioning

confidence: 99%

See 2 more Smart Citations

The Conflict Escalation Resolution (CONFER) Database

Georgakis

Panagakis

Zafeiriou

et al. 2017

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

Conflict is usually defined as a high level of disagreement taking place when individuals act on incompatible goals, interests, or intentions. Research in human sciences has recognized conflict as one of the main dimensions along which an interaction is perceived and assessed. Hence, automatic estimation of conflict intensity in naturalistic conversations would be a valuable tool for the advancement of human-centered computing and the deployment of novel applications for social skills enhancement including conflict management and negotiation. However, machine analysis of conflict is still limited to just a few works, partially due to an overall lack of suitable annotated data, while it has been mostly approached as a conflict or (dis)agreement detection problem based on audio features only. In this work, we aim to overcome the aforementioned limitations by a) presenting the Conflict Escalation Resolution (CONFER) Database, a set of excerpts from audio-visual recordings of televised political debates where conflicts naturally arise, and b) reporting baseline experiments on audio-visual conflict intensity estimation. The database contains approximately 142 minutes of recordings in Greek language, split over 120 non-overlapping episodes of naturalistic conversations that involve two or three interactants. Subject-and session-independent experiments are conducted on continuous-time (frame-by-frame) estimation of real-valued conflict intensity, as opposed to binary conflict/non-conflict clas- * Corresponding author. E-mail address: christos.georgakis@imperial.ac.uk Preprint submitted to Image and Vision Computing December 20, 2016 A C C E P T E D M A N U S C R I P T ACCEPTED MANUSCRIPTsification. For the problem at hand, the efficiency of various audio and visual features and fusion of them as well as various regression frameworks is examined. Experimental results suggest that there is much room for improvement in the design and development of automated multi-modal approaches to continuous conflict analysis. The CONFER Database is publicly available for non-commercial use at http://ibug.doc.ic.ac.uk/resources/confer/.

show abstract

Section: Featuresmentioning

confidence: 99%

Section: Accepted M Manuscriptmentioning

confidence: 99%

Section: Evaluation Metricsmentioning

confidence: 99%

See 1 more Smart Citation

The Conflict Escalation Resolution (CONFER) Database

Georgakis

Panagakis

Zafeiriou

et al. 2017

Image and Vision Computing

Self Cite

View full text Add to dashboard Cite

show abstract

“…(26) The respective closed-form solutions are obtained by substituting (25) and (26) into (23) or (24).…”

Section: Discussionmentioning

confidence: 99%

Discriminant Incoherent Component Analysis

Georgakis

Panagakis

Pantić

2016

IEEE Trans. on Image Process.

Self Cite

View full text Add to dashboard Cite

Abstract-Face images convey rich information which can be perceived as a superposition of low-complexity components associated with attributes, such as facial identity, expressions and activation of facial action units (AUs). For instance, low-rank components characterizing neutral facial images are associated with identity, while sparse components capturing non-rigid deformations occurring in certain face regions reveal expressions and action unit activations. In this paper, the Discriminant Incoherent Component Analysis (DICA) is proposed in order to extract lowcomplexity components corresponding to facial attributes, which are mutually incoherent among different classes (e.g., identity, expression, AU activation) from training data, even in the presence of gross sparse errors. To this end, a suitable optimization problem, involving the minimization of nuclear-and 1-norm, is solved. Having found an ensemble of class-specific incoherent components by the DICA, an unseen (test) image is expressed as a group-sparse linear combination of these components, where the non-zero coefficients reveal the class(es) of the respective facial attribute(s) that it belongs to. The performance of the DICA is experimentally assessed on both synthetic and real-world data. Emphasis is placed on face analysis tasks, namely joint face and expression recognition, face recognition under varying percentages of training data corruption, subject-independent expression recognition, and action unit detection by conducting experiments on 4 datasets. The proposed method outperforms all the methods that is compared to in all tasks and experimental settings.

show abstract

“…The problem of continuous pain estimation has been addressed in [17,18,24,29,32,42]. Kaltwang et al proposed a three step approach to pain estimation.…”

Section: Automatic Pain Recognitionmentioning

confidence: 99%

Cumulative attributes for pain intensity estimation

Egede

Valstar

2017

Proceedings of the 19th ACM International Conference on Multimodal Interaction

View full text Add to dashboard Cite

Pain estimation from face video is a hard problem in automatic behaviour understanding. One major obstacle is the difficulty of collecting sufficient amounts of data, with balanced amounts of data for all pain intensity levels. To overcome this, we propose to adopt Cumulative Attributes, which assume that attributes for high pain levels with few examples are a superset of all attributes of lower pain levels. Experimental results show a consistent relative performance increase in the order of 20% regardless of features used. Our final system significantly outperforms the state of the art on the UNBC McMaster Shoulder Pain database by using cumulative attributes with Relevance Vector Regression on a combination of features, including appearance, geometric, and deep learned features.

show abstract

Doubly Sparse Relevance Vector Machine for Continuous Facial Behavior Estimation

Cited by 46 publications

References 50 publications

The Conflict Escalation Resolution (CONFER) Database

The Conflict Escalation Resolution (CONFER) Database

Discriminant Incoherent Component Analysis

Cumulative attributes for pain intensity estimation

Contact Info

Product

Resources

About