Towards large scale multimedia indexing

Le, Nam; Bredin, Hervé; Sargent, Gabriel; India, Miquel; López-Otero, Paula; Barras, Claude; Guinaudeau, Camille; Gravier, Guillaume; Fonseca, Gabriel Barbosa da; Freire, Izabela Lyon; Patrocínio, Zenilton Kleber G. do; Guimarães, Silvio Jamil Ferzoli; Martí, Gerard; Morros, Josep Ramon; Hernando, Javier; Docío-Fernández, Laura; Garcı́a-Mateo, Carmen; Meignier, Sylvain; Odobez, Jean-Marc

doi:10.1145/3095713.3095732

Cited by 7 publications

(5 citation statements)

References 19 publications

(23 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SpeakerID is an important task for automatic organization of dialogue contents such as TV programs, radio podcasts, and online meetings and has gained significant research efforts [4,5,6,7,8]. Most previous work on SpeakerID approaches the task via multi-model setting, where the input to the models involve both videos/images and transcripts of the dialogues.…”

Section: Related Workmentioning

confidence: 99%

“…This process is crucial for enhancing the accessibility and searchability of multimedia content, enabling users to find segments featuring specific speakers. As a result, the development of effective SpeakerID systems has attracted significant research efforts, as evidenced by a body of work [4,5,6,7,8], striving to overcome the challenges associated with this complex task.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Calculation of stability constants of new metal-thiosemicarbazone complexes based on the QSPR modeling using MLR and ANN methods

Minh¹,

Minh²,

Van³

et al. 2021

DThU JoS

View full text Add to dashboard Cite

In this study, the stability constants (logβ11) of twenty-eight new complexes between several ion metals and thiosemicarbazone ligands were predicted on the basis of the quantitative structure property relationship (QSPR) modeling. The stability constants were calculated from the results of the QSPR models. The QSPR models were built by using the multivariate least regression (QSPRMLR) and artificial neural network (QSPRANN). The molecular descriptors, physicochemical and quantum descriptors of complexes were generated from molecular geometric structure and semi-empirical quantum calculation PM7 and PM7/sparkle. The best linear model QSPRMLR involves five descriptors, namely Total energy, xch6, xp10, SdsN, and Maxneg. The quality of the QSPRMLR model was validated by the statistical values that were R2train = 0.860, Q2LOO = 0.799, SE = 1.242, Fstat = 54.14 and PRESS = 97.46. The neural network model QSPRANN with architecture I(5)-HL(9)-O(1) was presented with the statistical values: R2train = 0.8322, Q2CV = 0.9935 and Q2test = 0.9105. Also, the QSPR models were evaluated externally and achieved good performance results with those from the experimental literature. In addition, the results from the QSPR models could be used to predict the stability constants of other new metal-thiosemicarbazones.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Calculation of stability constants of new metal-thiosemicarbazone complexes based on the QSPR modeling using MLR and ANN methods

Minh¹,

Minh²,

Van³

et al. 2021

DThU JoS

View full text Add to dashboard Cite

show abstract

“…Other face recognition methods (even those formulated in a Bayesian framework) [33,34,9,28,18], often limit themselves to point estimates of parameters and predictions, occasionally including ad-hoc confidence metrics. A distinct advantage of our approach is that it is probabilistic end-to-end, and thus naturally provides predictions with principled, quantifiable uncertainties.…”

Section: Related Workmentioning

confidence: 99%

“…Typical application domains involve the annotation of personal photo galleries [33,34,3,13], multimedia (e.g. TV) [28,18] or security/surveillance [17]. Our work focuses on egocentric human-like face recognition, a setting which seems largely unexplored, as most of the work using first-person footage appears to revolve around other tasks like object and activity recognition, face detection, and tracking [4].…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

From Face Recognition to Models of Identity: A Bayesian Approach to Learning About Unknown Identities from Unsupervised Data

Castro

Nowozin

2018

Computer Vision – ECCV 2018

View full text Add to dashboard Cite

Current face recognition systems robustly recognize identities across a wide variety of imaging conditions. In these systems recognition is performed via classification into known identities obtained from supervised identity annotations. There are two problems with this current paradigm: (1) current systems are unable to benefit from unlabelled data which may be available in large quantities; and (2) current systems equate successful recognition with labelling a given input image. Humans, on the other hand, regularly perform identification of individuals completely unsupervised, recognising the identity of someone they have seen before even without being able to name that individual. How can we go beyond the current classification paradigm towards a more human understanding of identities? We propose an integrated Bayesian model that coherently reasons about the observed images, identities, partial knowledge about names, and the situational context of each observation. While our model achieves good recognition performance against known identities, it can also discover new identities from unsupervised data and learns to associate identities with different contexts depending on which identities tend to be observed together. In addition, the proposed semi-supervised component is able to handle not only acquaintances, whose names are known, but also unlabelled familiar faces and complete strangers in a unified framework.Work done during an internship at Microsoft Research.

show abstract

Hierarchical multi-label propagation using speaking face graphs for multimodal person discovery

Fonseca

Sargent

Sicre

et al. 2020

Multimed Tools Appl

Self Cite

View full text Add to dashboard Cite

TV archives are growing in size so fast that manually indexing becomes unfeasible. Automatic indexing techniques can be applied to overcome this issue, and this work proposes an unsupervised technique for multimodal person discovery. To achieve this goal, we propose a hierarchical label propagation technique based on quasi-flat zones theory, that learns from labeled and unlabeled data and propagates names through a multimodal graph representation. In this representation, we combine audio, video, and text processing techniques to model the data as a graph of speaking faces. In the proposed modeling, we extract names via optical character recognition and propagate them through the graph using audiovisual relationships between speaking faces. We also use a random walk label propagation and two graph clustering strategies to serve as baselines. The proposed label propagation techniques always outperform the clustering baselines on the quantitative assessments. Our approach also outperforms all literature methods tested on the same dataset except for one, which uses a different preprocessing step. The proposed hierarchical label propagation and the random walk baseline produce highly equivalent results according to the Kappa coefficient, but the hierarchical propagation is parameter-free and over 9 times faster than the random walk under the same configurations.

show abstract

Towards large scale multimedia indexing

Cited by 7 publications

References 19 publications

Calculation of stability constants of new metal-thiosemicarbazone complexes based on the QSPR modeling using MLR and ANN methods

Calculation of stability constants of new metal-thiosemicarbazone complexes based on the QSPR modeling using MLR and ANN methods

From Face Recognition to Models of Identity: A Bayesian Approach to Learning About Unknown Identities from Unsupervised Data

Hierarchical multi-label propagation using speaking face graphs for multimodal person discovery

Contact Info

Product

Resources

About