VAST contest dataset use in education

Whiting, Mark; North, Chris; Endert, Alex; Scholtz, Jean; Haack, Jereme; Varley, Carrie; Thomas, Jim

doi:10.1109/vast.2009.5333245

Cited by 16 publications

(18 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The contest datasets have been used in class projects 13 and lend themselves for evaluation studies since answers can be easily graded because of the embedded ground truth. However, the contest datasets are too large and too complex to be used in lab experiments for formal evaluations of visual analytics systems.…”

Section: Using Contest Datasets For Evaluationmentioning

confidence: 99%

“…Our analysis activities in the VAST Contest/ Challenges using the large examples with embedded ground truth exposed a number of shortcomings in the Jigsaw system, and thus, the activities functioned very much in a formative evaluation sense. The contest datasets have been used in class projects 13 and lend themselves for evaluation studies since answers can be easily graded because of the embedded ground truth. However, the contest datasets are too large and too complex to be used in lab experiments for formal evaluations of visual analytics systems.…”

Section: Using Contest Datasets For Evaluationmentioning

confidence: 99%

See 1 more Smart Citation

Reflections on the evolution of the Jigsaw visual analytics system

Görg¹,

Liu

Stasko

2013

Information Visualization

View full text Add to dashboard Cite

Analyzing and understanding collections of textual documents is an important task for professional analysts and a common everyday scenario for nonprofessionals. We have developed the Jigsaw visual analytics system to support these types of sensemaking activities. Jigsaw's development benefited significantly from the existence of the VAST Contest/Challenge that provided (1) diverse document collections to use as examples, (2) controlled exercises with a set of analytic tasks and solutions for judging results, and (3) visibility and publicity to help communicate our ideas to others. This article describes our participation in a series of VAST Contest/Challenge efforts and how this participation helped influence Jigsaw's design and development. We describe how the system's capabilities have evolved over time, and we identify the particular lessons that we learned by participating in the challenges.

show abstract

Section: Using Contest Datasets For Evaluationmentioning

confidence: 99%

Section: Using Contest Datasets For Evaluationmentioning

confidence: 99%

Reflections on the evolution of the Jigsaw visual analytics system

Görg¹,

Liu

Stasko

2013

Information Visualization

View full text Add to dashboard Cite

show abstract

“…We also collected feedback from users through email, which confirmed that datasets are often used to teach classes [18] and various other purposes. We asked eleven questions about the VAST Challenge and its impact on their goals for their research.…”

Section: Surveysmentioning

confidence: 83%

A reflection on seven years of the VAST challenge

Scholtz

Whiting

Plaisant

et al. 2012

Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors - Novel Evaluation Methods for Visualization

Self Cite

View full text Add to dashboard Cite

We describe the evolution of the IEEE Visual Analytics Science and Technology (VAST) Challenge from its origin in 2006 to present (2012).The VAST Challenge has provided an opportunity for visual analytics researchers to test their innovative thoughts on approaching problems in a wide range of subject domains against realistic datasets and problem scenarios. Over time, the Challenge has changed to correspond to the needs of researchers and users. We describe those changes and the impacts they have had on topics selected, data and questions offered, submissions received, and the Challenge format.

show abstract

“…Moreover, the use of datasets and problems is expanding into the educational realm. 29 The enduring central presence of the VAST Challenge at the conference, the sustained rate of participation and dataset downloads, and the participant survey results indicate continuing interest from the researchers and practitioners and suggests that the Challenge is contributing to the overall evaluation of visual analytics systems. Evaluating accuracy remains the main challenge as we move toward more realistic scenarios and data.…”

Section: Resultsmentioning

confidence: 98%

Evaluation of visual analytics environments: The road to the Visual Analytics Science and Technology challenge evaluation methodology

Scholtz

Plaisant

Whiting

et al. 2013

Information Visualization

Self Cite

View full text Add to dashboard Cite

Evaluation of software can take many forms ranging from algorithm correctness and performance to evaluations that focus on the value to the end user. This article presents a discussion of the development of an evaluation methodology for visual analytics environments. The Visual Analytics Science and Technology Challenge was created as a community evaluation resource. This resource is available to researchers and developers of visual analytics environments and allows them to test out their designs and visualization and compare the results with the solution and the entries prepared by others. Sharing results allows the community to learn from each other and to hopefully advance more quickly. In this article, we discuss the original challenge and its evolution during the 7 years since its inception. While the Visual Analytics Science and Technology Challenge is the focus of this article, there are lessons for many involved in setting up a community evaluation program, including the need to understand the purpose of the evaluation, decide upon the right metrics to use, and the appropriate implementation of those metrics including datasets and evaluators. For ongoing evaluations, it is also necessary to track the evolution and to ensure that the evaluation methodologies are keeping pace with the science being evaluated. The discussions on the Visual Analytics Science and Technology Challenge on these topics should be pertinent to many interested in community evaluations.

show abstract

VAST contest dataset use in education

Cited by 16 publications

References 11 publications

Reflections on the evolution of the Jigsaw visual analytics system

Reflections on the evolution of the Jigsaw visual analytics system

A reflection on seven years of the VAST challenge

Evaluation of visual analytics environments: The road to the Visual Analytics Science and Technology challenge evaluation methodology

Contact Info

Product

Resources

About