A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation

Goutte, Cyril; Gaussier, Éric

doi:10.1007/978-3-540-31865-1_25

Cited by 1,504 publications

(871 citation statements)

References 7 publications

(12 reference statements)

Supporting

Mentioning

718

Contrasting

Unclassified

Order By: Relevance

“…The test set was compared to the reference set using the weighted harmonic mean of precision and recall (F1), a recognised standard test for measuring performance of information retrieval methods (Goutte and Gaussier, 2005). Table 1 summarises the results of the comparison between manual GOA annotation and our predicted annotation and shows precision, recall and F-Score for the three GO ontology categories.…”

Section: Evaluation Of Annotation Methodsmentioning

confidence: 99%

Graph-based sequence annotation using a data integration approach

Pesch

Lysenko

Hindle

et al. 2008

Journal of Integrative Bioinformatics

View full text Add to dashboard Cite

SummaryThe automated annotation of data from high throughput sequencing and genomics experiments is a significant challenge for bioinformatics. Most current approaches rely on sequential pipelines of gene finding and gene function prediction methods that annotate a gene with information from different reference data sources. Each function prediction method contributes evidence supporting a functional assignment. Such approaches generally ignore the links between the information in the reference datasets. These links, however, are valuable for assessing the plausibility of a function assignment and can be used to evaluate the confidence in a prediction. We are working towards a novel annotation system that uses the network of information supporting the function assignment to enrich the annotation process for use by expert curators and predicting the function of previously unannotated genes. In this paper we describe our success in the first stages of this development. We present the data integration steps that are needed to create the core database of integrated reference databases (UniProt, PFAM, PDB, GO and the pathway database AraCyc) which has been established in the ONDEX data integration system. We also present a comparison between different methods for integration of GO terms as part of the function assignment pipeline and discuss the consequences of this analysis for improving the accuracy of gene function annotation. The methods and algorithms presented in this publication are an integral part of the ON-DEX system which is freely available from http://ondex.sf.net/.

show abstract

Section: Evaluation Of Annotation Methodsmentioning

confidence: 99%

Graph-based sequence annotation using a data integration approach

Pesch

Lysenko

Hindle

et al. 2008

Journal of Integrative Bioinformatics

View full text Add to dashboard Cite

show abstract

“…Clusters linked with no reference tree are classified as false positive (FP). We can evaluate the detection accuracy in terms of "recall" (r = TP/(TP + FN)), which indicates the tree detection rate, and "precision" (p = TP/(TP + FP)), which indicates the correctness of the detected trees [31]. Table 3 shows the accuracy assessments for trees located in different forest storeys within six test subplots with ID of DHS0101, DHS0102, DHS0201, DHS0202, DHS0301 and DHS0302.…”

Section: Performance Evaluationmentioning

confidence: 99%

Adaptive Mean Shift-Based Identiﬁcation of Individual Trees Using Airborne LiDAR Data

Chen

2017

Remote Sensing

View full text Add to dashboard Cite

Abstract:Identifying individual trees and delineating their canopy structures from the forest point cloud data acquired by an airborne LiDAR (Light Detection And Ranging) has significant implications in forestry inventory. Once accurately identified, tree structural attributes such as tree height, crown diameter, canopy based height and diameter at breast height can be derived. This paper focuses on a novel computationally efficient method to adaptively calibrate the kernel bandwidth of a computational scheme based on mean shift-a non-parametric probability density-based clustering technique-to segment the 3D (three-dimensional) forest point clouds and identify individual tree crowns. The basic concept of this method is to partition the 3D space over each test plot into small vertical units (irregular columns containing 3D spatial features from one or more trees) first, by using a fixed bandwidth mean shift procedure and a small square grouping technique, and then rough estimation of crown sizes for distinct trees within a unit, based on an original 2D (two-dimensional) incremental grid projection technique, is applied to provide a basis for dynamical calibration of the kernel bandwidth for an adaptive mean shift procedure performed in each partition. The adaptive mean shift-based scheme, which incorporates our proposed bandwidth calibration method, is validated on 10 test plots of a dense, multi-layered evergreen broad-leaved forest located in South China. Experimental results reveal that this approach can work effectively and when compared to the conventional point-based approaches (e.g., region growing, k-means clustering, fixed bandwidth or multi-scale mean shift), its accuracies are relatively high: it detects 86 percent of the trees ("recall") and 92 percent of the identified trees are correct ("precision"), showing good potential for use in the area of forest inventory.

show abstract

“…For classification tasks, we used the terms "true positives", "true negatives", "false positives" (type I error), and "false negatives" (type II error) to compare the results of the classifier against the gold standard (Goutte & Gaussier, 2005). The terms "positive" and "negative" refer to the result indicated by the classifier, whereas the terms "true" and "false" refer to whether that result corresponds to the gold standard.…”

Section: Sentiment-driven Feedback In Inter-editor Communicationmentioning

confidence: 99%

The Impact of Sentiment-driven Feedback on Knowledge Reuse in Online Communities

Grigore¹,

Rosenkranz²,

Sutanto³

2015

THCI

View full text Add to dashboard Cite

Abstract:Knowledge reuse is of increasing importance for organizations. Despite the extant research, we still do not adequately understand the ways peers are motivated to reuse knowledge with the help of wiki technologies. In this paper, we study the motivation for knowledge reuse in a prominent instance of online social production: Wikipedia. Studying knowledge reuse in Wikipedia is important since Wikipedia has been able to leverage the benefits of efficient knowledge reuse to produce knowledge goods of relatively high quality. Specifically, we explore: 1) how Wikipedia editors (any peer who contributes to developing articles in Wikipedia) communicate their feedback toward each other's work in peer conversations and 2) to what extent sentiment-driven feedback impacts the level of knowledge reuse in Wikipedia. The results show that displaying sentiment-driven feedback positively influenced the level of knowledge reuse. Our study further shows a significant difference in the level of knowledge reuse between editors who shared mainly positive or mainly negative sentiments. Specifically, displaying mainly positive feedback corresponded to a superior level of knowledge reuse than displaying mainly negative feedback. We contribute to the extant literature of online social production communities in general and Wikipedia in particular by providing a first building block for research on peer feedback's role in developing and sustaining wiki-based knowledge reuse. We discuss our findings' implications for theory and practice.

show abstract

A Probabilistic Interpretation of Precision, Recall and F-Score, with Implication for Evaluation

Cited by 1,504 publications

References 7 publications

Graph-based sequence annotation using a data integration approach

Graph-based sequence annotation using a data integration approach

Adaptive Mean Shift-Based Identiﬁcation of Individual Trees Using Airborne LiDAR Data

The Impact of Sentiment-driven Feedback on Knowledge Reuse in Online Communities

Contact Info

Product

Resources

About