Podium: Ranking Data Using Mixed-Initiative Visual Analytics

Abstract

People often rank and order data points as a vital part of making decisions. Multi-attribute ranking systems are a common tool used to make these data-driven decisions. Such systems often take the form of a table-based visualization in which users assign weights to the attributes representing the quantifiable importance of each attribute to a decision, which the system then uses to compute a ranking of the data. However, these systems assume that users are able to quantify their conceptual understanding of how important particular attributes are to a decision. This is not always easy or even possible for users to do. Rather, people often have a more holistic understanding of the data. They form opinions that data point A is better than data point B but do not necessarily know which attributes are important. To address these challenges, we present a visual analytic application to help people rank multi-variate data points. We developed a prototype system, Podium, that allows users to drag rows in the table to rank order data points based on their perception of the relative value of the data. Podium then infers a weighting model using Ranking SVM that satisfies the user's data preferences as closely as possible. Whereas past systems help users understand the relationships between data points based on changes to attribute weights, our approach helps users to understand the attributes that might inform their understanding of the data. We present two usage scenarios to describe some of the potential uses of our proposed technique: (1) understanding which attributes contribute to a user's subjective preferences for data, and (2) deconstructing attributes of importance for existing rankings. Our proposed approach makes powerful machine learning techniques more usable to those who may not have expertise in these areas.

Interactive Learning for Identifying Relevant Tweets to Support Real-time Situational Awareness

“…However, these tools do not employ machine learning for relevance classification and do not integrate user feedback to improve their underlying models or algorithms. Visual analytics has also been increasingly used to improve various machine learning processes, such as feature selection [13], attribute weighting [48], and labeling [8,9,18], and even understanding the models themselves [22,39,42]. Sacha et al [40] proposed a framework to discuss the various forms of human interaction with machine learning models in visual analytics systems and theorized that VA tools could increase knowledge and usability of machine learning components.…”

Githru: Visual Analytics for Understanding Software Development History Through Git Metadata Analysis

“…The scope of the grouping is confined to similar commits in each stem to preserve the temporal sequence and topology. We exploit the Simple Additive Weighting (SAW) model in calculating similarity since this model is intuitive for users to understand and is known to serve exploration well [69,74]. To measure similarity, we choose five criteria from the commit metadata (i.e., author, commit date, commit type, file, and message).…”

Investigating Direct Manipulation of Graphical Encodings as a Method for User Interaction

“…However, some researchers have also enabled direct manipulation of graphical encodings in other visualization types. This includes the direct manipulation of: position of cells in table visualizations to either steer the underlying ranking model [46] or explore rankings [31,44]; angle of a pie chart segment to navigate the time dimension [20]; rows and columns in matrix visualizations [30,42]; nodes in tree visualizations [45]; and position of tokens in unit based visualizations [16].…”