A Framework for Evaluating Snippet Generation for Dataset Search

Abstract

Reusing existing datasets is of considerable significance to researchers and developers. Dataset search engines help a user find relevant datasets for reuse. They can present a snippet for each retrieved dataset to explain its relevance to the user's data needs. This emerging problem of snippet generation for dataset search has not received much research attention. To provide a basis for future research, we introduce a framework for quantitatively evaluating the quality of a dataset snippet. The proposed metrics assess the extent to which a snippet matches the query intent and covers the main content of the dataset. To establish a baseline, we adapt four state-of-the-art methods from related fields to our problem, and perform an empirical evaluation based on real-world datasets and queries. We also conduct a user study to verify our findings. The results demonstrate the effectiveness of our evaluation framework, and suggest directions for future research.

Towards Ontology Reshaping for KG Generation with User-in-the-Loop: Applied to Bosch Welding

“…This can be addressed by uniform interpolation, also known as forgetting [28,31], which we also plan to study in the future. Furthermore, we plan to compare our work with the body of research, where we actively contributed, on ontology evolution [12,53,54], knowledge modelling and summarisation [8,9,21,30,[49][50][51], ontology extraction or bootstrapping [17,36], and to investigate how to extend our work to account for ontology aggregation techniques [3,24], and to develop end-user interfaces for exploration and improvement of reshaped ontologies [1,2,20,38,[41][42][43].…”

SemML: Facilitating Development of ML Models for Condition Monitoring with Semantics

“…In particular, we envision to improve the usability by exploring keyword- [80][81][82] and faceted-search [83][84][85][86] over catalogues of ML pipe-lines, as well as with summarisation techniques for ontologies, Knowledge Graphs, and semantically represented ML-pipelines [87][88][89] and possibly auto-generated text enhancements for ML-pipelines [90,91]. We also consider a possibility to offer dataset search accompanying SemML to help users to select the most appropriate datasets for concrete ML tasks [92][93][94][95]. Moreover, we consider developing further visual paradigms and interfaces to improve users experience when interacting with SemML [17,18,96,97].…”

Explaining Documents' Relevance to Search Queries

“…It was also called query-biased summarization by Tombros and Sanderson [68]. Snippet generation is an active area of research and has been studied in the context of Web search [74], XML retrieval [24], semantic search [78], and more recently dataset search [79]. Early Web search engines presented query-independent snippets consisting of the first tokens of the result document.…”