To ensure accountability and mitigate harm, it is critical that diverse stakeholders can interrogate black-box automated systems and find information that is understandable, relevant, and useful to them. In this paper, we eschew prior expertise-and role-based categorizations of interpretability stakeholders in favor of a more granular framework that decouples stakeholders' knowledge from their interpretability needs. We characterize stakeholders by their formal, instrumental, and personal knowledge and how it manifests in the contexts of machine learning, the data domain, and the general milieu. We additionally distill a hierarchical typology of stakeholder needs that distinguishes higher-level domain goals from lower-level interpretability tasks. In assessing the descriptive, evaluative, and generative powers of our framework, we find our more nuanced treatment of stakeholders reveals gaps and opportunities in the interpretability literature, adds precision to the design and comparison of user studies, and facilitates a more reflexive approach to conducting this research.
The Visualization for Cyber Security research community (VizSec) addresses longstanding challenges in cyber security by adapting and evaluating information visualization techniques with application to the cyber security domain. This research effort has created many tools and techniques that could be applied to improve cyber security, yet the community has not yet established unified standards for evaluating these approaches to predict their operational validity. In this paper, we survey and categorize the evaluation metrics, components, and techniques that have been utilized in the past decade of VizSec research literature. We also discuss existing methodological gaps in evaluating visualization in cyber security, and suggest potential avenues for future research in order to help establish an agenda for advancing the state-of-the-art in evaluating cyber security visualizations.
Large general-purposed community question-answering sites are becoming popular as a new venue for generating knowledge and helping users in their information needs. In this paper we analyze the characteristics of knowledge generation and user participation behavior in the largest question-answering online community in South Korea, Naver Knowledge-iN. We collected and analyzed over 2.6 million question/answer pairs from fifteen categories between 2002 and 2007, and have interviewed twenty six users to gain insights into their motivations, roles, usage and expertise. We find altruism, learning, and competency are frequent motivations for top answerers to participate, but that participation is often highly intermittent. Using a simple measure of user performance, we find that higher levels of participation correlate with better performance. We also observe that users are motivated in part through a point system to build a comprehensive knowledge database. These and other insights have significant implications for future knowledge generating online communities.
Before contributing new knowledge, individuals must attain requisite background knowledge or skills through schooling, training, practice, and experience. Given limited time, individuals often choose either to focus on few areas, where they build deep expertise, or to delve less deeply and distribute their attention and efforts across several areas. In this paper we measure the relationship between the narrowness of focus and the quality of contribution across a range of both traditional and recent knowledge sharing media, including scholarly articles, patents, Wikipedia, and online question and answer forums. Across all systems, we observe a small but significant positive correlation between focus and quality.
No abstract
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.
hi@scite.ai
10624 S. Eastern Ave., Ste. A-614
Henderson, NV 89052, USA
Copyright © 2024 scite LLC. All rights reserved.
Made with 💙 for researchers
Part of the Research Solutions Family.