Evaluation metrics for the practical application of URREF ontology: An illustration on data criteria

Villiers, J. P. de; Focke, Richard W.; Pavlin, Gregor; Jousselme, Anne-Laure; Dragos, Valentina; Laskey, Kathryn Blackmond; Costa, Paulo C. G.; Blasch, Erik

doi:10.23919/icif.2017.8009879

Cited by 21 publications

(6 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Deep learning (DL) has changed the data processing and fusion methods in the last decade, but explainability is needed for user adoption [58]. Since DL needs large data to train, uncertainty representations are needed for data evaluation [59]. DL black-box methods have challenged understanding of what is the data association; whereas successful output explainable decisions have shown promise.…”

Section: Deep Learning (Dl) -Analytics Without the Usermentioning

confidence: 99%

Value-based sensor and information fusion

Blasch

2024

Signal Processing, Sensor/Information Fusion, and Target Recognition XXXIII

View full text Add to dashboard Cite

There are many examples, methods, and processes showing the importance of sensor, data, and information fusion. However, there is a need to determine the value added of information fusion in the context of data (e.g., multimodal, amount, and resolution), software (e.g., artificial intelligence/machine learning), hardware (e.g., size, weight, and performance), as well as architectures (e.g., cloud, fog, and edge computing). This paper utilizes the analytical hierarchy process (AHP) to determine the pairwise performance needs among different human-machine information fusion system tradeoffs to show the value added from sensor fusion. The paper examines the potential value added by coordinating deep, active, and reinforcement learning for information fusion systems. Among the information metrics, the combined methods of artificial intelligence learning methods highlight user requirements for accuracy, confidence, and timeliness.

show abstract

Section: Deep Learning (Dl) -Analytics Without the Usermentioning

confidence: 99%

Value-based sensor and information fusion

Blasch

2024

Signal Processing, Sensor/Information Fusion, and Target Recognition XXXIII

View full text Add to dashboard Cite

show abstract

“…In the context of target tracking, a variety of evaluation metrics with physical significance have been proposed, which can evaluate the practicability of the tracking algorithm and the consistency of the expected and assessed results. These metrics can be divided into three categories: effectiveness, timeliness, and accuracy, which can be seen in [ 21 , 22 , 23 , 24 ]. This paper also followed this division criterion for convenience.…”

Section: A Classification Of the Comprehensive Evaluation Metricsmentioning

confidence: 99%

Performance Evaluation Metrics and Approaches for Target Tracking: A Survey

Yan

et al. 2022

Sensors

View full text Add to dashboard Cite

Performance evaluation (PE) plays a key role in the design and validation of any target-tracking algorithms. In fact, it is often closely related to the definition and derivation of the optimality/suboptimality of an algorithm such as that all minimum mean-squared error estimators are based on the minimization of the mean-squared error of the estimation. In this paper, we review both classic and emerging novel PE metrics and approaches in the context of estimation and target tracking. First, we briefly review the evaluation metrics commonly used for target tracking, which are classified into three groups corresponding to the most important three factors of the tracking algorithm, namely correctness, timeliness, and accuracy. Then, comprehensive evaluation (CE) approaches such as cloud barycenter evaluation, fuzzy CE, and grey clustering are reviewed. Finally, we demonstrate the use of these PE metrics and CE approaches in representative target tracking scenarios.

show abstract

“…URREF criteria have generic definitions and can be instantiated for applications with coarse or finer granularity levels: evaluation metrics can be defined for data analysis [31], or more particularity for data specific types [32] or attributes: reliability and credibility [33], trust and self-confidence [34] or veracity [35]. While allowing a continuous analysis of uncertainty representation, quantification and evaluation [36], UR-REF criteria are detailed enough to capture model-embedded uncertainties [37], their propagation in the context of the decision loop [38] and offer a basis to compare different fusion methods [39].…”

Section: B Urref Ontology and Sources Of Uncertaintymentioning

confidence: 99%

Use cases for social data analysis with URREF criteria

Laudy

Dragos

2020

2020 IEEE 23rd International Conference on Information Fusion (FUSION)

View full text Add to dashboard Cite

Social data analysis has gained prominence in a wide range of domains as it provides users with the opportunity to communicate and share posts and topics. Automated analysis and reasoning about such data potentially derive meaningful insights, with tremendous potential for applications. However, the sheer volume, noise, and high dynamics of social data impose challenges that hinder the efficacy of algorithms. Automated approaches and classification models require then significant resources to be developed and prove to be often relevant to only a limited number of tasks. Imperfections of inputs, precision of techniques and accuracy of results need to be accounted and assessed as the process runs. This paper discuses two use cases allowing the investigation of implicit and explicit uncertainty arising when processing data gleaned on social media. The objective of this paper is twofold. The first objective is to set up the ETUR use case on social media analysis by adopting two tasks on opinion mining for cyberspace surveillance and information extraction for crisis analysis, respectively. The second objective is to discuss an overall methodology allowing the identification and assessment of uncertainties underlying each task The paper introduces two illustrations of social data analysis, investigates various sources of uncertainty and describes a methodology to select criteria for uncertainty assessment.

show abstract

Evaluation metrics for the practical application of URREF ontology: An illustration on data criteria

Cited by 21 publications

References 12 publications

Value-based sensor and information fusion

Value-based sensor and information fusion

Performance Evaluation Metrics and Approaches for Target Tracking: A Survey

Use cases for social data analysis with URREF criteria

Contact Info

Product

Resources

About