Toolkit research plays an important role in the field of HCI, as it can heavily influence both the design and implementation of interactive systems. For publication, the HCI community typically expects toolkit research to include an evaluation component. The problem is that toolkit evaluation is challenging, as it is often unclear what 'evaluating' a toolkit means and what methods are appropriate. To address this problem, we analyzed 68 published toolkit papers. From our analysis, we provide an overview of, reflection on, and discussion of evaluation methods for toolkit contributions. We identify and discuss the value of four toolkit evaluation strategies, including the associated techniques that each employs. We offer a categorization of evaluation strategies for toolkit researchers, along with a discussion of the value, potential limitations, and trade-offs associated with each strategy.
Media research has shown that people enjoy watching television as a part of socializing in groups. However, many constraints in daily life limit the opportunities for doing so. The Social TV project builds on the increasing integration of television and computer technology to support sociable, computer-mediated group viewing experiences. In this paper, we describe the initial results from a series of studies illustrating how people interact in front of a television set. Based on these results, we propose guidelines as well as specific features to inform the design of future "social television" prototypes.
International audienceMakers participate in remixing culture by drawing inspiration from, combining, and adapting designs for physical objects. To examine how makers remix each others' designs on a community scale, we analyzed metadata from over 175,000 digital designs from Thingiverse, the largest online design community for digital fabrication. Remixed designs on Thingiverse are predominantly generated designs from Customizer– a built-in web app for adjusting parametric designs. However, we find that these designs do not elicit subsequent user activity and the authors who generate them tend not to contribute additional content to Thingiverse. Outside of Customizer, influential sources of remixing include complex assemblies and design primitives, as well as non-physical resources posing as physical designs. Building on our findings, we discuss ways in which online maker communities could become more than just design repositories and better support collaborative remixing
We discuss touch-based navigation of 3D visualizations in a combined monoscopic and stereoscopic viewing environment. We identify a set of interaction modes, and a workflow that helps users transition between these modes to improve their interaction experience. In our discussion we analyze, in particular, the control-display space mapping between the different reference frames of the stereoscopic and monoscopic displays. We show how this mapping supports interactive data exploration, but may also lead to conflicts between the stereoscopic and monoscopic views due to users' movement in space; we resolve these problems through synchronization. To support our discussion, we present results from an exploratory observational evaluation with domain experts in fluid mechanics and structural biology. These experts explored domain-specific datasets using variations of a system that embodies the interaction modes and workflows; we report on their interactions and qualitative feedback on the system and its workflow.
Figure 1. Physical visualizations created with our fabrication tool, MakerVis: a) a layered scatterplot modeled after Rosling ; b) a prism map showing relative happiness in US states computed from Twitter sentiments; c), d), e) layered bar charts and line charts crafted by end users.
Existing research methods are largely intended to be conducted colocated and synchronously with a study population, but this approach is not feasible with remote or distributed populations. We describe a needs assessment study we conducted on Facebook. We report on our use of adapted versions of commonly used HCI research methods and lessons learned from this approach.
Existing research methods are largely intended to be conducted co-located and synchronously with a study population, but this approach is not feasible with remote or distributed populations. We describe a needs assessment study we conducted on Facebook. We report on our use of adapted versions of commonly used HCI research methods and lessons learned from this approach.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.