Are Data Sets Like Documents?: Evaluating Similarity-Based Ranked Search over Scientific Data

Abstract

The past decade has seen a dramatic increase in the amount of data captured and made available to scientists for research. This increase amplifies the difficulty scientists face in finding the data most relevant to their information needs. In prior work, we hypothesized that Information Retrieval-style ranked search can be applied to data sets to help a scientist discover the most relevant data amongst the thousands of data sets in many formats, much like text-based ranked search helps users make sense of the vast number of Internet documents. To test this hypothesis, we explored the use of ranked search for scientific data using an existing multi-terabyte observational archive as our test-bed. In this paper, we investigate whether the concept of varying relevance, and therefore ranked search, applies to numeric data-that is, are data sets are enough like documents for Information Retrieval techniques and evaluation measures to apply? We present a user study that demonstrates that data set similarity resonates with users as a basis for relevance and, therefore, for ranked search. We evaluate a prototype implementation of ranked search over data sets with a second user study and demonstrate that ranked search improves a scientist's ability to find needed data.

Aurum: A Data Discovery System

“…A true primary key will have a uniqueness ratio of 1, but because the profiler uses the Hyperloglog sketch to compute this, we may have a small error rate, so we simply check that the value is close to 1. When we retrieve the content-similar candidates, we iterate over them (line 18) and check whether they are PK/FK candidates (line 19), in which case we add the candidate PK/FK relationship to the EKG (20). This method is similar to PowerPivot's [7] approach, and works well in practice, as shown in the evaluation.…”

Demonstrating "Data Near Here"

“…T d behind the oth our similarity fu data within the uch summarizati search engine ty between the g [14]. The result ity [14,15] We separately t poral, and envi es from a prior the dataset con that study. We s Figure 5: in eac oint-wise calcul culated from the scores for tempo atial search term in Figure 5c.…”

“…With correlat search term tes n is reasonably v imilar ranking. W of using a co ance is reasonabl rsus applying th f the bubble re in both scoring other high-scor and scores calc ons in the datase ested search ter ironmental varia study [15]. We ntents to the 2,9 show the results ch case we grap lation (on the v e dataset summ oral search term ms in Figure 5 The results for d n. Finally, we co ms for each searc g orientation of most cases, the ween the two app ficient of determ mbined scores.…”

“…DNH usability and effectiveness was shown in two user studies. Megler and Maier [15] initially evaluated the appropriateness of a proposed similarity measure, based in cognitive science, via a user study that tested whether a population of scientists had a relatively consistent mental model of what it meant for a collection of numeric data to be "close" to a numeric search term. They found that the population of scientists surveyed provided markedly similar responses, and concluded that it might be possible to replicate the scientists' assessments.…”

Data Discovery and Reuse in Data Service Practices: A Global Perspective

“…From user perspectives, research has revealed the similarities and differences between dataset search and document retrieval (Kern & Mathiak, 2015;Megler & Maier, 2015), and recognized that locating data for reuse in research is challenging for many (Gregory et al, 2020). Specifically, data discovery and reuse are purpose-driven activities, characterized by a variety of data needs and discovery strategies (Gregory et al, 2020b) that are interwoven with processes of sensemaking (Koesten et al, 2020).…”