Correlated Chained Gaussian Processes for Datasets With Multiple Annotators

Abstract

The labeling process within a supervised learning 1 task is usually carried out by an expert, which provides the 2 ground truth (gold standard) for each sample. However, in many 3 real-world applications, we typically have access to annotations 4 provided by crowds holding different and unknown expertise 5 levels. Learning from crowds intends to configure machine 6 learning paradigms in the presence of multi-labelers, residing on 7 two key assumptions: the labeler's performance does not depend 8 on the input space, and independence among the annotators 9 is imposed. Here, we propose the correlated chained Gaussian 10 processes from multiple annotators-(CCGPMA) approach, which 11 models each annotator's performance as a function of the input 12 space and exploits the correlations among experts. Experimental 13 results associated with classification and regression tasks show 14 that our CCGPMA performs better modeling of the labelers' 15 behaviour, indicating that it consistently outperforms other state-16 of-the-art learning from crowds approaches.

Chained Deep Learning Using Generalized Cross-Entropy for Multiple Annotators Classification

“…Here, the simulations are performed by assuming: (i) dependencies among annotators and (ii) the labelers’ performances are modeled as a function of the input features. In turn, the semiparametric latent factor model is used to build the labels, as follows [ 56 ]: Define Q deterministic functions and their combination parameters . Compute , where is the n -th component of , being the 1-D representation of the input features in by using t-SNE approach [ 29 ].…”