Combining User Reputation and Provenance Analysis for Trust Assessment

Abstract

Trust is a broad concept that in many systems is often reduced to user reputation alone. However, user reputation is just one way to determine trust. The estimation of trust can be tackled from other perspectives as well, including by looking at provenance. Here, we present a complete pipeline for estimating the trustworthiness of artifacts given their provenance and a set of sample evaluations. The pipeline is composed of a series of algorithms for (1) extracting relevant provenance features, (2) generating stereotypes of user behavior from provenance features, (3) estimating the reputation of both stereotypes and users, (4) using a combination of user and stereotype reputations to estimate the trustworthiness of artifacts, and (5) selecting sets of artifacts to trust. These algorithms rely on the W3C PROV recommendations for provenance and on evidential reasoning by means of subjective logic. We evaluate the pipeline over two tagging datasets: tags and evaluations from the Netherlands Institute for Sound and Vision’s Waisda? video tagging platform, as well as crowdsourced annotations from the Steve.Museum project. The approach achieves up to 85% precision when predicting tag trustworthiness. Perhaps more importantly, the pipeline provides satisfactory results using relatively little evidence through the use of provenance.

Towards web documents quality assessment for digital humanities scholars

“…More extensively, Zaveri et al [19] provide a review on quality assessment for Linked Data. We also investigated the assessment of crowdsourced annotations using provenance analysis [6,17]. These methods will be adopted also in this context.…”

A User Centred Perspective on Structured Data Discovery

“…At 28% this attribute was ranked much higher than other quality dimensions such as accuracy, completeness, timeliness and cleanliness, which are in the focus of many quality repair approaches (Wand and Wang, 1996). The importance of provenance resonates with previous work in data quality (Ceolin et al, 2016;Malaverri et al, 2013); there is also a large body of literature proposing frameworks and tools to capture and use provenance, though their use in practice is not widespread (for example Stamatogiannakis et al (2014); Simmhan et al (2008)).…”

A Case Study of Using Analytic Provenance to Reconstruct User Trust in a Guided Visual Analytics System

“…Other work looked at combining provenance information with different types of metrics. To estimate trust, Ceolin et al [10] proposed to combine analytic provenance with user reputation. They built and evaluated a computational pipeline whereby relevant provenance features are extracted, then used to generate 'stereotypes' of user behaviour.…”