The VAST Challenge: history, scope, and outcomes: An introduction to the Special Issue

Cook, Kristin A.; Grinstein, Georges; Whiting, Mark

doi:10.1177/1473871613490678

Cited by 37 publications

(21 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Plaisant et al [22] also recommend using benchmarks and contests to perform insight-based evaluations. These benchmarks have been used in contests that were popular in the InfoVis Community in 2003-2005, and then moved to the VAST Community with the VAST Challenge 2 organized every year since 2006 [5,23]. These contests remain a remarkable asset of the visualization community to share evaluation results and collect realistic benchmarks to test systems and engage researchers.…”

Section: Efforts In the Visualization Communitymentioning

confidence: 99%

Evaluating Visual Data Analysis Systems

Battle

Angelini

Binnig

et al. 2018

Proceedings of the Workshop on Human-in-the-Loop Data Analytics

View full text Add to dashboard Cite

Visual data analysis is a key tool for helping people to make sense of and interact with massive data sets. However, existing evaluation methods (e.g., database benchmarks, individual user studies) fail to capture the key points that make systems for visual data analysis (or visual data systems) challenging to design. In November 2017, members of both the Database and Visualization communities came together in a Dagstuhl seminar to discuss the grand challenges in the intersection of data analysis and interactive visualization. In this paper, we report on the discussions of the working group on the evaluation of visual data systems, which addressed questions centered around developing better evaluation methods, such as "How do the different communities evaluate visual data systems?" and "What we could learn from each other to develop evaluation techniques that cut across areas?". In their discussions, the group brainstormed initial steps towards new joint evaluation methods and developed a first concrete initiative-a trace repository of various real-world workloads and visual data systems-that enables researchers to derive evaluation setups (e.g., performance benchmarks, user studies) under more realistic assumptions, and enables new evaluation perspectives (e.g., broader meta analysis across analysis contexts, reproducibility and comparability across systems).

show abstract

Section: Efforts In the Visualization Communitymentioning

confidence: 99%

Evaluating Visual Data Analysis Systems

Battle

Angelini

Binnig

et al. 2018

Proceedings of the Workshop on Human-in-the-Loop Data Analytics

View full text Add to dashboard Cite

show abstract

“…In their case, analysts found self‐organized fund‐raising behaviors in the Haiti Earthquake through multiple combinations of filters. The IEEE Visual Analytics Challenge (IEEE VAST Challenge) has provided several challenges in crisis management with social media data [CGW14]. For example, in 2014, they required participants to analyze a kid‐nap event and several crisis issues in Mini‐Challenge 3.…”

Section: Systems and Applicationsmentioning

confidence: 99%

Social Media Visual Analytics

Chen

Lin

Yuan

2017

Computer Graphics Forum

View full text Add to dashboard Cite

With the development of social media (e.g. Twitter, Flickr, Foursquare, Sina Weibo, etc.), a large number of people are now using them and post microblogs, messages and multi‐media information. The everyday usage of social media results in big open social media data. The data offer fruitful information and reflect social behaviors of people. There is much visualization and visual analytics research on such data. We collect state‐of‐the‐art research and put it into three main categories: social network, spatial temporal information and text analysis. We further summarize the visual analytics pipeline for the social media, combining the above categories and supporting complex tasks. With these techniques, social media analytics can apply to multiple disciplines. We summarize the applications and public tools to further investigate the challenges and trends.

show abstract

“…Challenges such as YOHO [3], MNIST [4], HPC Challenge [5], ImageNet [6] and VAST [7], [8] have played important roles in driving progress in fields as diverse as machine learning, high performance computing and visual analytics. YOHO is the Linguistic Data Consortium database for voice verification systems and has been a critical enabler of speech research.…”

Section: Introductionmentioning

confidence: 99%

Static graph challenge: Subgraph isomorphism

Samsi

Gadepally

Hurley

et al. 2017

2017 IEEE High Performance Extreme Computing Conference (HPEC)

View full text Add to dashboard Cite

The rise of graph analytic systems has created a need for ways to measure and compare the capabilities of these systems. Graph analytics present unique scalability difficulties. The machine learning, high performance computing, and visual analytics communities have wrestled with these difficulties for decades and developed methodologies for creating challenges to move these communities forward. The proposed Subgraph Isomorphism Graph Challenge draws upon prior challenges from machine learning, high performance computing, and visual analytics to create a graph challenge that is reflective of many real-world graph analytics processing systems. The Subgraph Isomorphism Graph Challenge is a holistic specification with multiple integrated kernels that can be run together or independently. Each kernel is well defined mathematically and can be implemented in any programming environment. Subgraph isomorphism is amenable to both vertex-centric implementations and array-based implementations (e.g., using the Graph-BLAS.org standard). The computations are simple enough that performance predictions can be made based on simple computing hardware models. The surrounding kernels provide the context for each kernel that allows rigorous definition of both the input and the output for each kernel. Furthermore, since the proposed graph challenge is scalable in both problem size and hardware, it can be used to measure and quantitatively compare a wide range of present day and future systems. Serial implementations in C++, Python, Python with Pandas, Matlab, Octave, and Julia have been implemented and their single threaded performance have been measured. Specifications, data, and software are publicly available at GraphChallenge.org.

show abstract

The VAST Challenge: history, scope, and outcomes: An introduction to the Special Issue

Cited by 37 publications

References 16 publications

Evaluating Visual Data Analysis Systems

Evaluating Visual Data Analysis Systems

Social Media Visual Analytics

Static graph challenge: Subgraph isomorphism

Contact Info

Product

Resources

About