Figure 1: Refinery allows exploration of large, heterogeneous networks by visualizing subgraphs of items relevant to user queries. Here, querying for a paper in a publication network has led this user to a conference session of possible interest.
AbstractBrowsing is a fundamental aspect of exploratory information-seeking. Associative browsing represents a common and intuitive set of exploratory strategies in which users step iteratively from familiar to novel bits of information. In this paper, we examine associative browsing as a strategy for bottom-up exploration of large, heterogeneous networks. We present Refinery, an interactive visualization system informed by guidelines for associative browsing drawn from literature on exploratory information-seeking. These guidelines motivate Refinery's query model, which allows users to simply and expressively construct queries using heterogeneous sets of nodes. This system computes degree-of-interest scores for associated content using a fast, random-walk algorithm. Refinery visualizes query nodes within a subgraph of results, providing explanatory context, facilitating serendipitous discovery, and stimulating continued exploration. A study of 12 academic researchers using Refinery to browse publication data demonstrates how the system enables discovery of valuable new content, even within existing areas of expertise.
Crowdsourcing is a common strategy for collecting the "gold standard" labels required for many natural language applications. Crowdworkers differ in their responses for many reasons, but existing approaches often treat disagreements as "noise" to be removed through filtering or aggregation. In this paper, we introduce the workflow design pattern of crowd parting: separating workers based on shared patterns in responses to a crowdsourcing task. We illustrate this idea using an automated clustering-based method to identify divergent, but valid, worker interpretations in crowdsourced entity annotations collected over two distinct corpora -Wikipedia articles and Tweets. We demonstrate how the intermediatelevel view provide by crowd-parting analysis provides insight into sources of disagreement not easily gleaned from viewing either individual annotation sets or aggregated results. We discuss several concrete applications for how this approach could be applied directly to improving the quality and efficiency of crowdsourced annotation tasks.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.